I can relate to a lot of what you say. I do technically have friends, but they're either from university and live in England, or online friends, so no one I can physically hang out with. I'm also stuck at home a lot and have had a lot of trouble learning to function on my own, as even when I've had friends, they've been the ones to initiate/organise things.
One thing I'll tell you is I wouldn't worry about the "nerdy" aspect too much. I'm into video games, metal, a little bit of anime occasionally, roleplaying games, things of that general nature. Very few of my friends have those same interests. Even the ones that do like one or two of those things, they aren't really the focus of our relationship. My ex, for example, was into all the same stuff as me (to an even larger extent in fact) and while it was awesome... it really didn't amount to all that much in practice. If you told me now that I'd never hook up with a girl who I had so much "in common" with in terms of shared interests, I wouldn't even really care. If anything, it became boring after a while to have every silent void filled with needlessly in depth analysis of anime or video games, so weirdly, it can even be a hindrance now and again.
I started making friends when I stopped worrying so much about finding people who are just like me and getting accepted for who I am. I know this is a massive stereotype, but in my experience, it is somewhat true that "nerdy" people are more introverted and closed off, and thus will make little effort to make friends with you unless they are just as lonely. I spent a year going to university socities such as the anime society we had, and I didn't make a single friend, and only had a handful of conversations. But when I just tried to be more sociable and friendly to the people around me, dare to make more jokes in public and be open minded to new things, rather than just set in my "identity" (but that's a whole other rant I have on modern society lol), I began to make a lot of friends and be well liked by most everyone.
The way I look at it, it IS awesome to have someone who you can do things you are legitimately passionate about with, no question about it. But ultimately, if you enjoy someone's company, if you connect with them on a deeper level, you don't really give a fuck what you're doing together, since it's the fact that you're doing it with that friend that brings you joy. Conversely, you can have everything in common with them, all your favourite shows and bands and whatnot, but if you just don't really click, none of that's gonna make a difference. So you might as well just focus on being nice and open to anyone you meet, and eventually, you'll make some friends. Not to mention, you can always be introduced to new things, as well as introduce your friends to the things you like.
With all that said, this is just my personal experience. There's probably just as many people who will tell you the exact opposite. But ultimately, it is about figuring out what works for you, and for that you need to put yourself out there and try different things. Best of luck, you seem pretty cool, so I'm sure it'll work out!