We have a webapp which exposes a REST interface. Json is used as the data format for most of the apis. 98% of those json messages were less than 500 Kb. But 2% of them can go above 100 MB. It was observed that after processing one such 100 MB json message, the process memory went up to 500 MB and stayed there. It never came down even after running the webapp for hours and processing small json messages. Interesting observation is that memory remained at 500 MB even after processing multiple 100 MB json messages. On analyzing the problem it was found that 'json.loads' is the culprit. Calling
The test code shown below simulates the situation .If the call to json.loads is omitted then this issue is not observed. GC counts printed after invocation of jsontest is (150, 9, 0) which indicates that gc threshold (700, 10, 10) is not met.