The Imperial House Law was determined in 1889, when the Constitution of the Empire of Japan was promulgated. The former established the constitutional monarchy while the latter dealt with the internal matters of the Imperial House. They were both supreme rules in their respective spheres and established was the principle that internal matters of the Imperial House were decided by the Imperial House itself without any procedures in the Diet. The Imperial Household Ministry was to deal with the Imperial House's internal matters and it was confirmed in an Imperial Order in 1908 that the Imperial Household Minister, the Head of the Ministry, assisted the Emperor in all matters concerning the Imperial House.

After World War ?? the Imperial Household Ministry was downsized, the Imperial Household Ministry became the Imperial Household Office and was placed under the Prime Minister of Japan with the implementation of the constitution of Japan on May 3, 1947.

On June 1, 1949 when the Law Establishing the Prime Minister's Office came into force, the Imperial Household Office became the Imperial Household Agency as an external agency of the Prime Minister's Office.
Since January 6, 2001 the Imperial Household Agency has been placed under the Cabinet Office, in accordance with the Law Establishing the Cabinet Office.