In 1908, a group of local women protested the dumping of Berkeley garbage in their community. Armed with two shotguns and a twenty-two-caliber rifle, they confronted the drivers of the wagons near what is now the corner of San Pablo Avenue and Buchanan Street. The women told the drivers of the horse-drawn garbage wagons to go home, which they did quickly and without complaint. Shortly thereafter, the residents of the town voted to incorporate as the City of Ocean View. In 1909, voters changed the name of the city, primarily to distinguish the city from the adjacent section of Berkeley which had previously been named Ocean View. On a vote of 38 to 6 the city was renamed in honor of Albany, New York, the birthplace of the city's first mayor, Frank Roberts.
According to the United States Census Bureau, the city has a total area of 5.5 square miles (14.2 km²), of which, 1.8 square miles (4.6 km²) of it is land and 3.7 square miles (9.5 km²) of it (67.28%) is water.