What Are The Rules For Refinancing?

That's simple. There aren't any rules. There are just some suggestions. You typically need to save at least one point from your mortgage rate to justify the refinancing fees. You typically should plan to stay in your home for at least another three years. Will you recoup your refinancing fees within two years?

From the Los Angeles Times:

Another factor to consider is how much longer you'll remain in debt with a new loan and how close you are to retirement. Ideally, you'll want to be mortgage-free by the time you quit work.

When you're just a few years into your loan, as you are, this is less an issue than if you've paid down your mortgage for five or more years. In the latter case, you should either consider opting for a shorter loan — 15 or 20 years, say — or make extra payments on a 30-year loan if you otherwise wouldn't pay off the mortgage by the time you're ready to retire.