Does it matter if we harvest something earlier than the connecting plots if we're trying to crossbreed them? In other words, if out of 8 patches, 4 of them finish a day before the others, would it affect my attempts at getting seeds from the remaining 4 if I harvested the others earlier?

I think I did do that once, but I don't recall getting seeds from the late bloomers.