The most likely cause is genetics. You might try and sus an environmental reason, but finicky genetics = bad gentetics, so really the same issue. Ruined? Possibly. You will probably find seeds in your crop if you continue this grow. From my experience, early pollination can be less...