You are basically just going into the cycle where you pay bills with the labor camps, destroy them to collect taxes, and then rebuy them to pay bills.
Back to the OP, the most simple concept to keep in mind to help guide anyone through the process when they get large enough to improvement swap is these three things:
1) You always want to have Labor Camps when you pay bills
2) You always want to have Factories when you buy infra
3) You always want to have the maximum amount of income boosters possible when you collect taxes.
I'm no longer sure at what infra level a person should start swapping. It used to be around 1400 infra, but with the changes in the price of Labor Camps and Factories I'm guessing it's closer to 1700-1800 infra. When I was between between 41 and 45 improvements, I quit destroying factories and instead swapped Labor Camps and intell agencies just because they were a little cheaper to replace. At 50 improvements you can compare your collections amount with buying a border wall or 2/3 of them after destroying your labor camps, but if you want to keep it simple you can just destroy the LC to collect and rebuy them to pay bills.