A native of Cordova, Tennessee, Al Brown was an outstanding athlete at Central High School before playing baseball for then Memphis State College and the Chicago White Sox minor league system. While coaching at Central, Overton High School and Memphis State, he was a part-time baseball scout for the Chicago White Sox organization until 1978.

While at Central as head baseball coach, his team won the State Championship in 1958. From 1959-1962, he served as Athletic Director and coached basketball and football at Overton High School in Memphis.

In 1962, Brown became the Head Baseball Coach and Assistant Basketball Coach at Memphis State University. During those days, he was the Color Commentator for the University of Memphis for basketball, football and the Memphis Chicks Double A Baseball with WMC-TV, WKNO-TV and Radio from 1963-1996. During this time, Brown authored two books, Coaching Isn't Easy and Basic Thoughts in Coaching Baseball.

Following his coaching career of 20 years, Brown was Director of the Liberty Bowl Memorial Stadium and the entire Fairgrounds for 20 years. He was a founder and served 14 years with the Mid-South Stadium Managers Association. He returned to the University of Memphis in 1991 to become the Executive Director of the University of Memphis M Club (former letter winners) for 15 years.

In 1984, Brown was awarded the Billy J. Murphy Award from the University of Memphis Athletic Department. He was inducted into the Memphis Amateur Sports Hall of Fame in 1990 and was awarded the Distinguished Service Award from the College Athletic Business Managers Association in 1991.

Brown is a past President and Director of the College Athletic Business Managers Association. He was President of the Memphis Amateur Sports Hall of Fame twice.

Brown has been a board member for the Tennessee Sports Hall of Fame for 26 years. He has served as President twice and had a hand in 42 nominations that became inductees of the TSHF Hall of Fame.