In 1924, two years before receiving semikhah, Ruderman married Faiga Kramer from a rabbinical family. In 1930, he joined his father-in-law, Rabbi Sheftel Kramer, at the latter's yeshiva in New Haven, Connecticut. In 1931, the Ruderman family moved to Cleveland, Ohio, where he served as one of the teachers at the Yeshiva.
In 1933, with his father-in-law's encouragement, Ruderman moved to Baltimore, where he was immediately offered a rabbinical post at Tiferes Yisroel Shul. Ruderman accepted the position on the condition that he be permitted to open a yeshiva using the synagogue's facilities. He began with six students and named the newly formed yeshiva Ner Yisroel. The yeshiva grew quickly, and Ruderman approached the renowned Rabbi Shimon Schwab, at the time rabbi of another Baltimore congregation, and invited him to join the faculty. Rabbi Schwab taught the first-year shiur in Ner Israel for several years, until he moved to Washington Heights. When Ruderman grew old, he became legally blind but could still read by holding a book within inches of his eyes that wore very thick glasses. He still held a siddur when davening. Ruderman led the yeshiva for 54 years until his death when Rabbi Shmuel Yaakov Weinberg, his son in law, took over. Ruderman was rosh yeshiva, while his brother-in-law, Rabbi Naftoli Neuberger took care ofthe financial side. Together, they built it into one of the largest yeshivas in America, producing thousands of rabbis, educators and learned laymen. Ruderman was also involved in many aspects of Jewish communal life outside of the Yeshiva. He was a member of the Council of Torah Sages of Agudath Israel and the chairman of the Rabbinic Advisory Board of Torah Umesorah.
Works
Around 1926, Ruderman published his only written work, Avodas Levi. The Sefer Avodath Levi Project to preserve his legacy has been in progress. Posthumously, his students have published several volumes of his teachings: ethical insights based on the weekly parsha named Sichos Levi, later re-written and republished as Sichos Avodas Levi, lectures on the 19th century work Minchas Chinuch and other Talmudic and halachic insights in Mas'as Levi, and lectures on Sukkah, Kiddushin, Kesubos, Bava Kamma, and Bava Metzia, as Shiurei Avodas Levi.
Death
Ruderman's death on July 11, 1987, the 14th of Tammuz, followed less than 18 months after the deaths of Rabbis Yaakov Kaminetsky and Moshe Feinstein. Ruderman was one of the last surviving rosh yeshiva who came to America from Lithuania early in the 20th century. His son-in-law, Rabbi Weinberg, who married his only child, Chana, succeeded him as rosh yeshiva of Ner Yisroel until Rabbi Weinberg's death in 1999. Weinberg's wife, Chana, died on January 23, 2012. Ruderman was buried in Baltimore's United Hebrew Cemetery.
