Caldwell was fired as head coach of the Indianapolis Colts two weeks ago in the wake of a 2-14 season and front office upheaval. He took over after Tony Dungy's retirement and in three seasons posted a mark of 26-22 with a trip to the Super Bowl after his first year.
Baltimore quarterback Joe Flacco had a banner year in 2011, throwing for 3,610 yards with 20 touchdowns and 12 interceptions as the Ravens won the AFC North.
Caldwell served as quarterbacks coach for the Colts in each of his first three seasons on Dungy's staff and was elevated to assistant head coach before the 2005 campaign. He was also on Dungy's staff for the 2001 season in Tampa Bay.
Prior to joining Dungy in Tampa, Caldwell was the head coach at Wake Forest from 1993-2000, guiding the Demon Deacons to a record of 26-63 with one winning season and one bowl appearance.