Not sure if it's completely related but I'm seeing slow builds because VS 2017 tries to rebuild the referenced projects of a plugin every single time I build that plugin. Even if no changes in the referenced plugins were made. I checked to make sure no files (images, views, etc...) had the "Copy Always" build option set to true. I'm not sure if this is a VS issue or an issue with how the solution is configured. I also made sure the Copy Local option of the reference project was set to False.
I've been experiencing long and painful startups in debugging mode. I've unloaded all the unnecessary project files and the build speed has improved significantly. When I start with debugging, it takes sometimes minutes to load. Most of the time is spent loading symbols and logging message from mainly EntityFramework.