Sorry you're having trouble getting this working.
Silverlight can indeed be a little more troublesome to get working but it should be possible.
For the crash, have you tried "rebuild with diagnostics"? This may give you a useful stack trace when it crashes to help point to the part that's going wrong.
For the other .xap files where it says it's not a valid assembly, you can read about how SA processes these here
- if you check the Manifest points to the right file, can you extract that and see if SA can process the DLL on its own? If you get the same message, has the DLL been postprocessed by some other tool at all?