Because it takes the ability of certain characters to teleport between continents to a whole new level. He is in Dorne brokering a deal and then 20 minutes later he's on a ship with Dany. It's reasonable to assume that each scene doesn't necessarily take place within the same time frame, but this was just silly.
Let's also not forget that Varys has never met Dany before. So him being on that ship in the background doesn't make that much sense even if he had been in Meereen already. What happened to the scene where they first meet and Varys has to explain to her why he was sending assassins after her in the first couple of seasons?
It's just lazy writing, again.