Consistently hitting .300 for the Lowell Textile Institute baseball teams that compiled a 21-21-2 record during his four years of play, Olson was the leader of the team from his catcher's position.  He was captain of the squad during the 1924 season and was named the team's Outstanding Player.  He caught the famous game against Holy Cross in April 1923, that ended a 5-5 tie despite the fact that Holy Cross had just returned from a southern trip and it was Textile's opening game.

Olson was a three-year letterman for the Textile football team, playing every position in the backfield.  He was captain of the 1923 gridiron squad that finished 4-4 and scored all 19 points in Textile's 19-0 win over Worcester Polytech.

Originally from Somerville, MA, Olson made as many contributions to the success of the Textile athletic program off the field as on.  He served as president of the Student-Athlete Council in 1923 and 1924 and was his class treasurer in 1923-24.  When the Textile baseball coach left in the middle of the season, Olson stepped in to fill the void through the remainder of the season.