After following the career of Supreme Court Justice Ruth Bader Ginsburg and observing her growing popularity among millennials, filmmakers Betsy West and Julie Cohen knew that the 85-year-old liberal justice had a story to tell.

But they weren’t expecting their documentary, “RBG,” to be holding its own during a crowded season at the box office.

Co-produced...