Here is how I see things playing out long term.
In the comics, at one point Ultron is defeated and is floating through space as just a radio wave of consciousness. He finds a techno-organic group known as the Phalanx and merges with them. The Phalanx play a role in GotG stuff, so may be relevant in GotG 2. Ultron then starts causing trouble in Kree space (and the full Avengers GotG crossover begins). During this massive battle, that sees Ultron nearly defeating the Kree, Iron Man and other Avengers help out.
As the movies go, I imagine it being more about Ultron being defeated by being teleported through a wormhole into space, using one of the infinity gems (three are still out there) and then being found by the Phalanx or possibly Adam Warlock (who Ultron temporarily controlled at one point) and starting his conquest of Kree space. I imagine Earth heroes won't show yet, but it will renew the attention of Thanos to Earth. Multiple infinity stones are known now and Thanos will begin his quest to claim them. One is in Asgard, the tesseract. When he claims that one (Thor 3?) Earth will be warned by Thor and the Guardians and Avengers will be joined together.
Meanwhile, back on Earth, in the aftermath of Ultron, Tony Stark realizes what he has wrought. He was the first masked hero. His presence triggered the rise of them and as Thor said, signaled to the rest of the universe that we were ready for a new level of war. Stark believes himself to blame for two massive tragedies. He takes the idea of tracking potential threats that SHIELD/Hydra were using in The Winter Soldier, and turning it into a registration protocol. Cap reiterates his "this isn't freedom. This is fear," argument and the two have a falling out. They fight and all goes to hell. This will allow the writers to adjust for any lost actor scenarios by killing off characters or having them leave the country. It is possible that the sudden appearance of an alien ship could stop all the fighting.
The wild cards: Marvel still has a number of unannounced films. Following Agents of SHIELD I cannot believe that The InHumans won't play a role. They even have the potential to let Coulson become Captain Marvel (Mah-Vel) or Skye to become Ms. Marvel, and possibly having her father be connected to the InHumans and/or Kree in some way. There are a lot of blanks to fill in, but Ultron was never a villain that just ceased to exist.