Īlthough there was a falling out between Rabbi Berlin and Rabbi Yosef Dov Soloveitchik, they ended up making peace and Rabbi Chaim Soloveitchik married Rabbi Berlin's granddaughter. A son from his first marriage, Chaim Berlin, became the rabbi of Moscow, a daughter married Rabbi Refael Shapiro, and his son from his second marriage was Rabbi Meir Berlin (later Bar-Ilan). His second wife was his niece, a daughter of Rabbi Yechiel Michel Epstein, the author of the Aruch haShulchan. His first wife was the daughter of Rabbi Yitzchok of Volozhin, the son of Rabbi Chaim Volozhin. Legend has it that he applied himself to his studies after overhearing his parents debating whether he should pursue a trade. However, most scholars believe this to be false.
According to some sources, Berlin initially was a weak student. His father Jacob, while not a rabbi, was a Talmudic scholar descendant of a German rabbinic family his mother was directly descended from Rabbi Meir Eisenstadt. Berlin was born in Mir, today in Belarus, in 1816 into a family of Jewish scholars renowned for its Talmudic scholarship.