Four of the banks - JPMorgan, Barclays, Citigroup and RBS - have agreed to plead guilty to US criminal charges.
The fifth, UBS, will plead guilty to rigging benchmark interest rates.
Barclays was fined the most, $2.4bn, as it did not join other banks in November to settle investigations by UK, US and Swiss regulators.
Barclays is also sacking eight employees involved in the scheme.
US Attorney General Loretta Lynch said that "almost every day" for five years from 2007, currency traders used a private electronic chat room to manipulate exchange rates.
Their actions harmed "countless consumers, investors and institutions around the world", she said.
Separately, the Federal Reserve fined a sixth bank, Bank of America, $205m over foreign exchange-rigging. All the other banks were fined by both the Department of Justice and the Federal Reserve.
Regulators said that between 2008 and 2012, several traders formed a cartel and used chat rooms to manipulate prices in their favour.
One Barclays trader who was invited to join the cartel was told: "Mess up and sleep with one eye open at night."
Several strategies were used to manipulate prices and a common scheme was to influence prices around the daily fixing of currency levels.
A daily exchange rate fix is held to help businesses and investors value their multi-currency assets and liabilities.
Until February, this happened every day in the 30 seconds before and after 16:00 in London and the result is known as the 4pm fix, or just the fix.
In a scheme known as "building ammo", a single trader would amass a large position in a currency and, just before or during the fix, would exit that position.
Other members of the cartel would be aware of the plan and would be able to profit.
"They engaged in a brazen 'heads I win, tails you lose' scheme to rip off their clients," said New York State superintendent of financial services Benjamin Lawsky.
The fines break a number of records. The criminal fines of more than $2.5bn are the largest set of anti-trust fines obtained by the Department of Justice.
The £284m fine imposed on Barclays by Britain's Financial Conduct Authority was a record by the regulator.
Meanwhile, the $925m fine imposed on Citigroup by the Department of Justice was the biggest penalty for breaking the Sherman Act, which covers competition law.
The guilty pleas from the banks are seen as highly significant as banks have settled previous investigations without an admission of guilt.
The Attorney General warned that further wrongdoing would taken extremely seriously: "The Department of Justice will not hesitate to file criminal charges for financial institutions that reoffend.
"Banks that cannot or will not clean up their act need to understand - it will be enforced."