Lit.: "Waterfall-mountain Castle"

Ooishi Sadashige, Musashi-no-Kami and vassal of the Yamanouchi-Uesugi Clan, built Takiyama Castle in 1521. In 1546, Houjou Ujiyasu took Kawagoe in a night battle (the Battle of Kawagoe Castle, destroying the Ougigayatsu-Uesugi Clan and erasing the influence of the Yamanouchi-Uesugi Clan in Musashi. Sadashige's son Ooishi Sadahisa adopted Houjou Ujiteru as his heir, then later became a vassal of the Houjou Clan. Ujiteru took over as the master of the castle and made massive repairs to it around 1558.

In 1569, Takeda Shingen's army of 20,000 camped to the north of the castle en route to an assault on Odawara Castle. A small force of 1,000 led by Oyamada Nobushige attacked from Kobotoke Ridge, penetrating all the way to the outermost walls, but a force of 2,000 Houjou troops staved off the attack.

However, following his near-defeat, Ujiteru decided that the castle's defenses were not optimal and abandoned it in favor of Hachiouji Castle.