Word of the Day: Bissextile
In the Roman calendar, on which ours is based, the year used to start in March. (Note that "September" means "seventh month," "October" means "eighth month," etc. That only makes sense if you start counting in March. Before the Caesars renamed July and August, they were Quintillus and Sextillus.) February was the last month of the year, which is why the extra day ended up there.
The calends was the first day of the ancient Roman month, from which days were counted backward to the ides, which was the 15th or 13th of the month, depending. In a leap year, the sixth day before the Calends of March (i.e. 24th February, counting back from the beginning of March) occurred twice. So February 25 was called a bissextile (twice sixth) day. That was the added day, not the last day of the month, although adding a day partway through the month gave that month a 29th day.
I feel sorry for February. With the Caesars hogging extra days for their months, even on a leap year it's the shortest month. I think the extra minutes, seconds and milliseconds should be hoarded up and, once every million years or so, February should have 32 days.