The assemblies should be built when the lib directory is build because the generated application manifest should be used with all the tools and executables that use the libraries.