Low Attack Speed on Crossbow

if you spam and keep track of time you should get roughly these numbers from ayumilove:
Strafe/Arrow Eruption/Blizzard/Iron Arrow/Normal Attack/Pierce
Faster (2): 630ms (-240ms)
Faster (3): 690ms (-180ms)
Fast (4): 720ms (-150ms)
Fast (5): 780ms (-90ms)
Normal (6): 870ms

 

unless you also had SI the speed should be different

 

No, what they're saying is that fast attack with booster and SI is the same as normal. SI is Speed Infusion and it's a seperate buff that increases your attack speed in the same way booster does.
Booster increases the speed by 2, Si increases the speed by 2, faster(2) is the fastest speed.
Fast(5) + booster = faster(3)
Fast(5) + booster + SI = faster(2) (because you can't go below 2)
Normal(6) + booster = fast(4)
Normal(6) + booster + SI = faster(2)

As you can see, normal(6) and fast(5) with only booster aren't the same, however, adding Speed Infusion will make them the same speed because of the maximum faster(2).

I hope this answers your question, identifying a difference of 60ms between cast speeds can be difficult. Perhaps if you'd like to doublecheck, you could set a timer and do 20 or 30 casts.