I've done some research into the best of the cheap/interesting/hole in the wall eats in NYC. This city has the most interesting food in the world and I am going to find it.
Here's the list I am going to start out with. If you know of some others, please post them under comments.
Caracas Arepa Bar (corn cakes stuffed with fillings like cheese, meat and plantains)
Chumley's (burgers and beer)
Corner Bistro (burgers)
Crif Dogs (hot dawgs)
Downtown Bakery II (mexican)
Fake Orchid (Thai)
Freemans (weird bistro w/ taxidermy)
Mama's Food Shop (comfort)
Pho Bang Restaurant
West Side Coffee Shop (mexican)
Yatagan Kebab House
Zaragoza Mexican Bakery (tamales)