Group Instance dropping frames by a lot!

I usually keep my project organized by having an assets scene and adding the assets via group instance in the levels.
Now I was doing one game asset directly in the scene and the performance was just fine around 60 fps.
After being almost finished, I copied it to my assets scene and loaded it again in my level via group instance.
The performance dropped by 15 fps!
How is this possible?
Seeing this makes me wonder if I should use group instance at all.
I always used them as a way to keep everything clean and organised.