Police called for the suspension of operations at the international airport, located near London's Docklands area, when workers at a building site around 1.5 miles away discovered the bomb, airport spokeswoman Jenny Lloyd said.
The site has been cordoned off and local train services have been suspended too, according to a spokeswoman for the Metropolitan Police who spoke on condition of anonymity in line with force policy.
The terminal building remained open, and airport officials asked passengers to check with airlines for further travel information. They provided no information about when flights were likely to resume.
The area where the bomb was discovered is in a part of London that was heavily bombed during World War II.
London City Airport served 2 million passengers last year.