A bunch of people talk about their guild like going through a dungeon like what do they mean are their like certain dungeons that you can pause or are they just really long and you stop once you get to a certain point?

A raid is different from a dungeon first off. Dungeons, to answer your question, have bosses and when you kill the final boss, it's finished. Raids take more players and you do the same thing, it's just harder.
People say that because they are running it with some guild members and are almost finished.

Short, sweet, and to the point.

you have to finish a dungeon to get valor points which in turn are used to buy new gear. and if you quit a dungeon to early you can get a penalty which wont allow you to join another one for 30 minutes.

Well, they could just be accomplashing the dungeon. Some dungeons are really long, but it depends on what you mean...if they say this for a week about the same dungeon then they are probably loco.