Fishon, both Newburgh and Kingston areas can be good in the spring.. Worms work, but for bigger fish, Live herring/eels are great.. Fresh herring chunks work well also, but you will be fighting off catfish and eels constantly ... bring some sabiki's and try to catch your own herring , as availabilty of fresh at bait shops is limited at best.. Gully's is a bar/resturant on the waterfront in Newburgh..
for bait, try Ceely's just to the south, off Rt. 9W.. Kingston is about 30 miles to the north and is where I do more of my river fishing.. The Rondout Creek, which flows into the Hudson at Kingston is a good place to try to catch herring, and offers some shore access.. but if ya strike out, Certified Marine on the south bank of the creek in Connelly sells bait.. including livies.
With all that said however, .I'm afraid you may have missed it this year.. The run usually starts around mid April, peaks most years around mid May and is usually done by the 1st week in June..this year things where a bit early, and with water temps pushing into the 70's here now, what few fish are left likely wont be sticking around much longer.. but let me know if you head up, and I'll be glad to try and point you in the right direction to get started