He began his practice in 1929. Around sometime in 1948, he got the coveted opportunity of working on the prosecution of Nathuram Godse who was the assasinator of Mahatma Gandhi. Subsequently, in 1949,he became the judge of Bombay High Court where he served for 10 years. In 1959,he was elevated to the Supreme Court as a judge. Finally, on 17th of December, 1970 he became the Chief Justice of India. However, he retired within a year, on 21st January 1971.
However, he again got an opportunity to serve for his country in the year 1977 when the home ministry appointed him as the head of a commission which was set up to inquire into the emergency situation in India at the time.This is regarded as one of his major contributions. The task of the commission was to look into the allegations of excesses, malpractices and misdeeds that might have occurred during the emergency due to the public authorities. It stated in its findings that Prime Minister Indira Gandhi was guilty of misusing the emergency provisions in order to stay in power after her conviction was passed by the Allahabad High Court for corrupt electoral practices.
He had clarified the objectives of the commission by stating that he was only going to look into the various circumstances that had transpired before the declaration of emergency to look into the veracity of the claim that emergency was declared upon valid grounds and not necessarily look into the violations that have taken place as a consequence of it.
He was also one of the fore most judges against whom impeachment motion had been moved. Nani Palkhivala, a legalstal wart had noted that the resolution passed by the lawyers association on Bombay for the removal of the judge was “distinct public disservice, instead of a public service, if they have created an impression in the public mind that our higher judiciary is not worthy of public confidence and respect.” In another instance, one government official, OP Gupta submitted to the Lok Sabha speaker G S Dhillon in May 1970, a motion to remove him from office based on certain remarks made by the judge. However, it was rejected and J.Shah continued in office.
Despite the inquiry report submitted by J. Shah, on December 28, 1979, Justice T.P.S. Chawla of the Delhi High Court passed the judgment quashing prosecution of Indira Gandhi.
In one curious instance, he was mentioned in fellow judge, MHidayatullah’s autobiography, “My own Boswell.” Though not directly, but indirectly J. Hidayatullah called J. Shah jealous of him.
However, he is regarded as one of the most dedicated judges towards work. He would talk less and would object to lack of ethics in his courtroom.However, he would do that with a sense of humour.