We developed a system called Life Game Orchestra that generates music by translating cellular patterns of Conway’s Game of Life into musical scales. A performer can compose music by controlling varying cell patterns and sounds with visual and auditory fun. A performer assigns the elements of tone to two-dimensional cell patterns in the matrix of the Game of Life. Our system searches defined cell patterns in the varying matrix dynamically. If the patterns are matched, corresponding tones are generated. A performer can make cells in the matrix by moving in front of a camera and interactively influencing the generation of music. The progress of the Game of Life is controlled with a clock defined by the performer to configure the groove of the music. By running multiple matrices with different pattern mapping, clock timing, and instruments, we can perform an ensemble. The Life Game Orchestra is a fusion system of the design of a performer and the emergence of cellular automata as a complex system.