• Wednesday's ruling came after a transgender woman filed a petition challenging the law.
• Human Rights Watch had called the law "abusive and outdated".
Japan's Supreme Court has ruled that it is unconstitutional to require citizens to be sterilised before they can officially change genders.
The 2004 law said people could only change their gender if they have no reproductive capacity.
Wednesday's ruling came after a transgender woman filed a petition challenging the law.
Human Rights Watch had called the law "abusive and outdated".
Wednesday's ruling reverses a 2019 verdict by the court which found the same law constitutional.
Japan is one of 18 countries that mandate the sterilisation surgery - a requirement that is also opposed by the World Health Organization. It is also the only Group of Seven (G7) nation that does not legally recognise same-sex unions.
The woman's lawyer had argued that her reproductive ability has already been diminished by years of hormone therapy, adding that surgery entailed physical suffering and the risk of after-effects.
Her request was denied by both the family and high court before she approached the Supreme Court.
"The government now needs to act quickly to remove the clause," Kanae Doi, Japan director of Human Rights Watch told news agency Reuters after the verdict. "It's late but never too late".
But some groups opposed to the law being changed had argued that if people were allowed to change their registered gender without surgery, it could result in women feeling unsafe. They also argued that it could cause legal confusion.
Recent opinion polls have shown growing support for LGBTQ-friendly laws - although there is opposition from conservative sections of society and politicians.
Earlier this month, a local family court ruled in favour of a transgender man - Gen Suzuki- who requested to have his gender legally changed without undergoing the surgery.
The family court judge, Takehiro Sekiguchi, said the current law violated Article 13 of the Constitution that stipulates all people shall be respected as individuals.