International Business Machine Corp.’s $33 billion bet on the world’s biggest Linux company shows how the once-maverick operating-system software has cemented its place at the heart of modern business computing.

Born 27 years ago out of an experiment by Finnish software developer Linus Torvalds, Linux is, technically speaking, the core operating system component that makes a computer work. Companies such as Red Hat Inc., which IBM is buying in its biggest acquisition ever, bundle this software with other free programs for...