I don't have a copy of Divi to test with, but I know I wasn't able to recreate any Internal Server Errors when I visited the Customizer with Twentysixteen active. I can't think of much that we run consistently, outside of register_post_type and register_taxonomy calls on init, so I'm not sure what could be causing the issue overall.
Do you have a dev site that you could use and deactivate/reactivate various things on without wrecking an active site?