This does not normally occur, so here are some things to consider:
Is the bore of the pulley worn?
Is the key, keyway in the crankshaft, or keyway in the pulley worn?
Is the crankshaft a smaller diameter than original?
Sometimes a burr may form at the bottom of the hole for a setscrew. ....This can prevent the setscrew from tightening against the shaft, yet the set screw appears to be tight.
If the lower pulley(s) are not in their proper position, they can cause misalignment of the belts, which causes a pulley to move.
If the engine shaft is parallel (in two planes) to the lower shaft, and the pulleys are properly aligned, there should be no force to cause the pulley to move. ....Rotational force will not cause a pulley to move, unless something is out of alignment.
To help eliminate movement of a pulley, a shallow depression (hole) drilled or filed into the shaft will accept the tip of the setscrew and prevent shifting of the pulley.