Yes, that's right. If you have the right cables, you can connect your USB flash drive or other USB device that has a driver for Windows CE installed. You need a micro-USB to mini-USB adapter as well as a micro-USB Host (OTG) adapter. You can search for these on Ebay or Google them. They are out there. I can't post links, here
If anyone knows of a micro-USB Host adapter, please post it, here! I've not found one.
You are limited to USB full speed (12 Mb/s raw bandwidth) and 100 mA power supply (so some USB flash drives might not work as they use more than that, but modern ones are fine). You can even plug in an ultrasound probe (Google "cell phone ultrasound") if you're so inclined (I am, you see).
This is not advertised as supported so I will take no responsibility for anything bad that happens, of course, but it works for me (the ultrasound probe and the flash drive) so I'm reporting the news. You can get a mouse driver for a few dollars, and I bet a keyboard simply works, too (although I've not tried it).
Enjoy.