Son of Usaburo Koyama, a samurai of the Oshi clan. He studied at the clan school Shinshukan and served as a teacher at Nagano Prefectural Normal School before joining the Ministry of Education in 1881. He became the principal of Koto Shogyo Gakko(later Hitotsubashi University) in 1895 and concurrently as the director of the Industrial School Bureau in January, 1898. He became the Vice-Minister of Education in May the same year only to retire from public office in just two months with the formation of the Okuma Kensei-to Cabinet. He became the president of Sanju-shi Ginko(the 34th Bank)in 1899. He assumed the chairman of the committee of Osaka Ginko(Osaka Bank) in 1901 and became a potentate in the Osaka financial community. He served as auditor of the South Manchuria Railways Company in 1915. He was selected as a member of the House of Peers in 1920.