I absolutely don't care about where a man chooses to put his wang, so long as it's a consenting place. It makes Hancock an absolutely awful person - and Coladangelo is also an absolutely awful person, because not only is she knowingly screwing a married man, she is also married, and there's six kids involved in all of this too, and there's a good bet this has been 20 years in the making - but that doesn't exactly mark him out as a government minister, or an MP, or even a public figure. He can bang as many consenting women, or men, or both, as he wants, with whatever kink he wants, so long as he can do the job objectively. The affair itself is a nothing professionally, even if it makes him (and her) a scumbag personally.
Efficacy of the COVID response aside (we're all second-guessing anyway, and largely after the fact, and we doubtless have less information than the SSHSC), it's pretty clear he cannot.
It's painfully obvious that he's ignored all the social distancing rules he's been foisting onto other people for over a year. Mixing fluids with your sidepiece is definitely mixing households, given that he's got a wife and three kids and she's got a husband and three kids. That's unacceptable, and he should have gone on that alone - not apologising for it and considering the matter closed. You can't establish a restrictive, authoritarian policy (and it is, even if it's an understandable and necessary one) for everyone else and piss all over it.
Having trysts with his mistress in his cabinet office is also on the scale of unacceptable and speaks of man with a schoolboy mentality and an inability to make sensible decisions if there's vagina involved, but making her an aide, giving her a parliamentary pass, and even having her accompany him to confidential meetings is just bald-faced nepotism - and the business with her brother looks to be pretty much the same. That's corruption, and it's ridiculous neither of them considered for a second how it would look if their affair ever came out; given that they're now leaving their respective spouses for each other anyway, it looks like the endgame of the affair was never even contemplated.
However... the biggest and most concerning part of all of it is how someone smuggled, fitted, and activated a hidden camera literally inside the parliamentary offices in Whitehall. It's an incredible breach of security and begs several further questions like "were there others", "what other information has it captured and how sensitive/secret is it", and "what if it had been a different kind of device, like the one that goes 'bang'?".