As long as you rinse it first, the only thing you have to worry about is the temperature change putting the fish into shock.
With mine, I rinsed it, boiled it, let it cool down for a while, then dumped it in. Very little clouding. Of course I used construction sand that had been sitting in my friends back yard for about 2 years under a tarp. Hey, you can't beat free.
Like I said earlier though, the only problems I had were some minor Ph fluctuations and a diatom bloom.
Another person on here recently posted a video of himself going through the whole process.
Fill a 5 gallon bucket half way with PFS, turn your garden hose on full , point it into the sand and let the bucket over flow. Stir the sand for little bit with your hands while its overflowing and that should get out most if the silt. Pour out the water and dump about half a gallon of sand in about ever 10 minute or so so as not to chill the tank out to much.