Ever since arch got an installer I’m not sure I’d consider it hard anymore. Still dumps you into a command line sure but it’s a long way away from the days of trying to figure out arcane partition block math
Each downtime is an opportunity to learn the weaknesses of your own system.
There are certain scenarios you have no control over (upstream problems), but others have contingencies. I enjoy working out these contingencies and determining whether the costs are worth the likelihoods - and even if they're not, that doesn't necessarily mean I won't cater for it.
For offline documentation, I use these in order of preference:
• Info¹ documentation, which I read directly in Emacs. (If you have ever used the terminal-based standalone “info” program, please try to forget all about it. Use Emacs to read Info documentation, and preferably use a graphical Emacs instead of a terminal-based one; Info documentation occasionally has images.)
• Gnome Devhelp².
• Zeal³
• RFC archive⁴ dumps provided by the Debian “doc-rfc“ package⁵.
I get why you want to self host, although I also get why you don’t want.
Selfhosting is a pain in the ass, it needs updating docker, things break sometimes, sometimes it’s only you and not anyone else so you’re left alone searching the solution, and even when it works it’s often a bit clunky.
I have a extremely limited list of self hosted tool that just work and are saving me time (first one on that list would be firefly) but god knows i wasted quite a bit of my time setting up stuffs that eventually broke and that i just abandoned.
Today I’m very happy with paying for stuff if the company is respecting privacy and has descent pricing.
There's your problem. Docker adds indirection on storage, networking, etc., and also makes upgrades difficult as you have to either rebuild the container, or rely on others to do so to get security and other updates.
If you stick to things that can be deployed as an upstream OS vendor package, or as a single binary (go-based projects frequently do this), you'll likely have a better time in the long run.
Maybe. There are pros and cons. Docker means you can run two+ different things on the same machine and update them separately. This is sometimes important when one project releases a feature you really want, while a different one just did a major update that broke something you care about. Running on the OS often means you have to update both.
Single binary sometimes works, but means you need more memory and disk space. (granted much less a concern today than it was back in 1996 when I first started self hosting, but it still can be an issue)
What project did you run into issues with? I've found any project that has gotten to the point of offering a Docker Compose seems to just work.
Plus I've found nearly every company will betray your trust in them at some point so why even give them the chance? I self host Home Assistant, but they seem to be the only company that actively enacts legal barriers for themselves so if Paulus gets hit by a bus tomorrow the project can't suddenly start going against the users.
> The premise is that by learning some of the fundamentals, in this case Linux, you can host most things yourself. Not because you need to, but because you want to, and the feeling of using your own services just gives you pleasure. And you learn from it.
Not only that, but it helps to eliminate the very real risk that you get kicked off of a platform that you depend on without recourse. Imagine if you lost your Gmail account. I'd bet that most normies would be in deep shit, since that's basically their identity online, and they need it to reset passwords and maybe even to log into things. I bet there are a non-zero number of HN commenters who would be fucked if they so much as lost their Gmail account. You've got to at least own your own E-mail identity! Rinse and repeat for every other online service you depend on. What if your web host suddenly deleted you? Or AWS? Or Spotify or Netflix? Or some other cloud service? What's your backup? If your answer is "a new cloud host" you're just trading identical problems.
My singular issue with self hosting specifically with email is not setting it up. Lots of documentation on setting up an email server.
But running it is different issue. Notably, I have no idea, and have not seen a resource talking about troubleshooting and problem solving for a self hosted service. Particularly in regards with interoperability with other providers.
As a contrived example, if Google blackballs your server, who do you talk to about it? How do you know? Do that have email addresses, or procedures for resolution in the error messages you get talking with them?
Or these other global, IP ban sites.
I’d like to see a troubleshooting guide for email. Not so much for the protocols like DKIM, or setting DNS up properly, but in dealing with these other actors that can impact your service even if it’s, technically, according to Hoyle, set up and configured properly.
Most services, including email providers, spam databases, and "ip-ban sites" have clear documentation, in terms of how to get on their good side, if needed, and it is often surprisingly straightforward to do so. Often it's as simple as filling out a relatively form.
Have you ever tried to use it? Because I fought for about 2 months with both Google and Microsoft, trying to self-host my mail server, to no success. The only answer was amongst the lines 'your server has not enough reputation'. Even though perfectly configured, DKIM, DMARC, etc. Now imagine a business not being able to send a message to anyone hosted on Gmail or Outlook, probably 80-90 percents of the companies out there.
They claim that, but everyone small I know who self hosted email has discovered that forms don't do anything. I switched to fastmail 15 years ago and my email got a lot better because they are big enough that nobody dares ignore them. (maybe the forms work better today than 15 years ago, but enough people keep complaining about this issue that I doubt it)
Own your own domain, point it to the email hosting provider of your choice, and if something went horribly wrong, switch providers.
Domains are cheap; never use an email address that's email-provider-specific. That's orthogonal to whether you host your own email or use a professional service to do it for you.
If doing so id also recommend not using the same email or domain for the registrar and for your email host…. If you are locked out of one you’d want to be able to access the other to change things.
As for the domain risks, my suggestions is to stick with the .com/.net/.org or something common in your country and avoid novelty ones such as .app, .dev, etc, even if you can't get the shortest and simpler name. And if you have some money to spare, just renew it to 10 years.
> I will lose some email history, but at least I don’t lose my email future.
I back up all my email every day, independent of my hosting provider. I have an automatic nightly sync to my laptop, which happens right before my nightly laptop backups.
Agreed. I’ve had the same email address for a decade now but cycled through the registrar’s email, Gmail, and M365 in that time. Makes it easy to switch.
Self hosting at home - what is higher risk? Your HDD dying or losing Gmail account?
Oh now you don’t only self host, now you have to have space to keep gear, plan backups, install updates, oh would be good to test updates so some bug doesn’t mess your system.
Oh you know installing updates or while backups are running it would be bad if you have power outage- now you need a UPS.
Oh you know what - my UPS turned out to be faulty and it f-up my HDD in my NAS.
No I don’t have time to deal with any of it anymore I have other things to do with my life ;)
Different strokes for different folks. Motivation for me has been a combination of independence and mistrust. Every single one of the larger tech companies have shown their priority to growth above making good products and services, and not being directly user hostile. Google search is worse now than it was 10 years ago. Netflix has ads with a paid subscription, so does YouTube. Windows is absolute joke, more and more we see user hostile software. Incentives aren’t aligned at all. As people who work in software, I get not wanting to do this stuff at home as well. But honestly I’m hoping for a future where a lot of these services can legit be self hosted by technical people for their local communities. Mastodon is doing this really well IMO. Self hosted software is also getting a lot easier to manage, so I’m quite optimistic that things will keep heading this way.
Note, I’ve got all the things you mentioned down to the UPSes setup in my garage, as well as multiple levels of backups. It’s not perfect, but works for me without much time input vs utility it provides. Each to their own.
If your trust is violated, typically the worst that happens is you are fed a couple more relevant ads or your data is used for some commercial purpose that has little to no effect on your life.
Is it really worth going through so much effort to mitigate that risk?
Again, it's a value judgement, so the answer is largely personal. For me, yes. The social license we give these larger companies after all the violated trust doesn't make sense. If your local shop owner/operator that you talked to everyday had the same attitude towards your when you went shopping and exchanged pleasantries with most weeks, people would confront them about their actions, and that shop wouldn't last long. We have created the disconnect for convenience, and tried to ignore the level of control these companies have on our day to day lives if they are so inclined or instructed to change their systems.
Cloud is just someone else's computer. These systems aren't special. Yes they are impressively engineered to deal with the scale they deal with, but when systems are smaller, they can get a lot simpler. I think as an industry we have conflated distributed systems with really hard engineering problems, when it really matter at what level of abstraction the distribution happens when it comes to down stream complexity.
It introduces some pretty important risks of its own though. If you accidentally delete/forget a local private key or lose your primary email domain there is no recourse. It's significantly easier to set up 2FA and account recovery on a third party service
Note that I'm not saying you shouldn't self-host email or anything else. But it's probably more risky for 99% of people compared to just making sure they can recover their accounts.
I have seen much more stories about people losing access to their Gmail because of a comment flagged somewhere else (i.e YouTube) than people losing access to their domains (it is hard to miss all these reminders about renewal and you shouldn't wait until then anyway so that's something under you control).
And good luck getting anyone from Google to solve your problem assuming you get to a human.
It's heartening in the new millennium to see some younger people show awareness of the crippling dependency on big tech.
Way back in the stone ages, before instagram and tic toc, when the internet was new, anyone having a presence on the net was rolling their own.
It's actually only gotten easier, but the corporate candy has gotten exponentially more candyfied, and most people think it's the most straightforward solution to getting a little corner on the net.
Like the fluffy fluffy "cloud", it's just another shrink-wrap of vendor lockin. Hook 'em and gouge 'em, as we used to say.
There are many ways to stake your own little piece of virtual ground. Email is another whole category. It's linked to in the article, but still uses an external service to access port 25. I've found it not too expensive to have a "business" ISP account, that allows connections on port 25 (and others).
Email is much more critical than having a place to blag on, and port 25 access is only the beginning of the "journey". The modern email "reputation" system is a big tech blockade between people and the net, but it can, and should, be overcome by all individuals with the interest in doing so.
I spend quite some years with linux systems, but i am using llms for configurating systems a lot these days. Last week i setup a server for a group of interns. They needed a docker kubernetes setup with some other tooling. I would have spend at least a day or two to set it up normally. Now it took maybe an hour. All the configurations, commands and some issues were solved with help of chatgpt. You still need to know your stuff, but its like having a super tool at hand. Nice.
Similarly, I was reconfiguring my home server and having Claude generate systemd units and timers was very handy. As you said you do need to know the material to fix the few mistakes and know what to ask for. But it can do the busywork of turning "I need this backup job to run once a week" into the .service and .timer file syntax for you to tweak instead of writing it from scratch.
Claude and others are still in the adoption phase so the services are good, and not user hostile as they will be in the extraction phase. Hopefully by then some agreement on how to setup RAG systems for actual human constructed documentation for these systems will be way more accessible, and have good results with much smaller self hosted models. IMO, this is where I think/hope the LLMs value to the average person will land long term. Search, but better at understanding the query. Sadly, they will also been used for a lot of user hostile nonsense as well.
I think it's just a turbo mode for figuring things out. Like posting to a forum and getting an answer immediately, without all those idiots asking you why you even want to do this, how software X is better than what you are using etc.
Obviously you should have enough technical knowledge to do a rough sanity check on the reply, as there's still a chance you get stupid shit out of it, but mostly it's really efficient for getting started with some tooling or programming language you're not familiar with. You can perfectly do without, it just takes longer. Plus You're not dependent on it to keep your stuff running once it's set up.
Ever since arch got an installer I’m not sure I’d consider it hard anymore. Still dumps you into a command line sure but it’s a long way away from the days of trying to figure out arcane partition block math
RIP "I use arch btw"
I self-host most of what I need but I recently faced the ultimate test when my Internet went down intermittently.
It raised some interesting questions:
- How long can I be productive without the Internet?
- What am I missing?
The answer for me was I should archive more documentation and NixOS is unusable offline if you do not host a cache (so that is pretty bad).
Ultimately I also found out self-hosting most of what I need and being offline really improve my productivity.
Each downtime is an opportunity to learn the weaknesses of your own system.
There are certain scenarios you have no control over (upstream problems), but others have contingencies. I enjoy working out these contingencies and determining whether the costs are worth the likelihoods - and even if they're not, that doesn't necessarily mean I won't cater for it.
I find that self hosting "devdocs" [1] and having zeal (on linux) [2] solves a lot of these problems with the offline docs.
[1] https://github.com/freeCodeCamp/devdocs
[2] https://zealdocs.org/
For offline documentation, I use these in order of preference:
• Info¹ documentation, which I read directly in Emacs. (If you have ever used the terminal-based standalone “info” program, please try to forget all about it. Use Emacs to read Info documentation, and preferably use a graphical Emacs instead of a terminal-based one; Info documentation occasionally has images.)
• Gnome Devhelp².
• Zeal³
• RFC archive⁴ dumps provided by the Debian “doc-rfc“ package⁵.
1. https://www.gnu.org/software/emacs/manual/html_node/info/
2. https://wiki.gnome.org/Apps/Devhelp
3. https://zealdocs.org/
4. https://www.rfc-editor.org/
5. https://tracker.debian.org/pkg/doc-rfc
I've taken this as far as I can. I love being disconnected from the internet for extended periods - they're my most productive times
I have a bash alias to use wget to recursively save full websites
yt-dlp will download videos you want to watch
Kiwix will give you a full offline copy of Wikipedia
My email is saved locally. I can queue up drafts offline
SingleFile extension will allow you to save single pages really effectively
Zeal is a great open source documentation browser
Could you share the bash alias? I would love this too.
https://srcb.in/nPU2jIU5Ca
Unfortunately it doesn't work well on single page apps. Let me know if anyone has a good way of saving those
I get why you want to self host, although I also get why you don’t want.
Selfhosting is a pain in the ass, it needs updating docker, things break sometimes, sometimes it’s only you and not anyone else so you’re left alone searching the solution, and even when it works it’s often a bit clunky.
I have a extremely limited list of self hosted tool that just work and are saving me time (first one on that list would be firefly) but god knows i wasted quite a bit of my time setting up stuffs that eventually broke and that i just abandoned.
Today I’m very happy with paying for stuff if the company is respecting privacy and has descent pricing.
> docker
There's your problem. Docker adds indirection on storage, networking, etc., and also makes upgrades difficult as you have to either rebuild the container, or rely on others to do so to get security and other updates.
If you stick to things that can be deployed as an upstream OS vendor package, or as a single binary (go-based projects frequently do this), you'll likely have a better time in the long run.
Maybe. There are pros and cons. Docker means you can run two+ different things on the same machine and update them separately. This is sometimes important when one project releases a feature you really want, while a different one just did a major update that broke something you care about. Running on the OS often means you have to update both.
Single binary sometimes works, but means you need more memory and disk space. (granted much less a concern today than it was back in 1996 when I first started self hosting, but it still can be an issue)
What project did you run into issues with? I've found any project that has gotten to the point of offering a Docker Compose seems to just work.
Plus I've found nearly every company will betray your trust in them at some point so why even give them the chance? I self host Home Assistant, but they seem to be the only company that actively enacts legal barriers for themselves so if Paulus gets hit by a bus tomorrow the project can't suddenly start going against the users.
> if the company is respecting privacy and has descent pricing.
Also an extremely limited list.
Can definitely become a trend given so many devs out there and so much that AI can produce at home which can be of arbitrary code quality…
> The premise is that by learning some of the fundamentals, in this case Linux, you can host most things yourself. Not because you need to, but because you want to, and the feeling of using your own services just gives you pleasure. And you learn from it.
Not only that, but it helps to eliminate the very real risk that you get kicked off of a platform that you depend on without recourse. Imagine if you lost your Gmail account. I'd bet that most normies would be in deep shit, since that's basically their identity online, and they need it to reset passwords and maybe even to log into things. I bet there are a non-zero number of HN commenters who would be fucked if they so much as lost their Gmail account. You've got to at least own your own E-mail identity! Rinse and repeat for every other online service you depend on. What if your web host suddenly deleted you? Or AWS? Or Spotify or Netflix? Or some other cloud service? What's your backup? If your answer is "a new cloud host" you're just trading identical problems.
My singular issue with self hosting specifically with email is not setting it up. Lots of documentation on setting up an email server.
But running it is different issue. Notably, I have no idea, and have not seen a resource talking about troubleshooting and problem solving for a self hosted service. Particularly in regards with interoperability with other providers.
As a contrived example, if Google blackballs your server, who do you talk to about it? How do you know? Do that have email addresses, or procedures for resolution in the error messages you get talking with them?
Or these other global, IP ban sites.
I’d like to see a troubleshooting guide for email. Not so much for the protocols like DKIM, or setting DNS up properly, but in dealing with these other actors that can impact your service even if it’s, technically, according to Hoyle, set up and configured properly.
Most services, including email providers, spam databases, and "ip-ban sites" have clear documentation, in terms of how to get on their good side, if needed, and it is often surprisingly straightforward to do so. Often it's as simple as filling out a relatively form.
Have you ever tried to use it? Because I fought for about 2 months with both Google and Microsoft, trying to self-host my mail server, to no success. The only answer was amongst the lines 'your server has not enough reputation'. Even though perfectly configured, DKIM, DMARC, etc. Now imagine a business not being able to send a message to anyone hosted on Gmail or Outlook, probably 80-90 percents of the companies out there.
They claim that, but everyone small I know who self hosted email has discovered that forms don't do anything. I switched to fastmail 15 years ago and my email got a lot better because they are big enough that nobody dares ignore them. (maybe the forms work better today than 15 years ago, but enough people keep complaining about this issue that I doubt it)
Own your own domain, point it to the email hosting provider of your choice, and if something went horribly wrong, switch providers.
Domains are cheap; never use an email address that's email-provider-specific. That's orthogonal to whether you host your own email or use a professional service to do it for you.
If doing so id also recommend not using the same email or domain for the registrar and for your email host…. If you are locked out of one you’d want to be able to access the other to change things.
This is my plan.
I will lose some email history, but at least I don’t lose my email future.
However, you can’t own a domain, you are just borrowing it. There is still a risk that gets shut down too, but I don’t think it is super common.
Why should you lose some email history? Just move the mails to a differente folder.
I self host my mails but still use a freemail for the contact address for my providers. No chicken and egg problem for me.
As for the domain risks, my suggestions is to stick with the .com/.net/.org or something common in your country and avoid novelty ones such as .app, .dev, etc, even if you can't get the shortest and simpler name. And if you have some money to spare, just renew it to 10 years.
> I will lose some email history, but at least I don’t lose my email future.
I back up all my email every day, independent of my hosting provider. I have an automatic nightly sync to my laptop, which happens right before my nightly laptop backups.
Agreed. I’ve had the same email address for a decade now but cycled through the registrar’s email, Gmail, and M365 in that time. Makes it easy to switch.
Self hosting at home - what is higher risk? Your HDD dying or losing Gmail account?
Oh now you don’t only self host, now you have to have space to keep gear, plan backups, install updates, oh would be good to test updates so some bug doesn’t mess your system.
Oh you know installing updates or while backups are running it would be bad if you have power outage- now you need a UPS.
Oh you know what - my UPS turned out to be faulty and it f-up my HDD in my NAS.
No I don’t have time to deal with any of it anymore I have other things to do with my life ;)
Different strokes for different folks. Motivation for me has been a combination of independence and mistrust. Every single one of the larger tech companies have shown their priority to growth above making good products and services, and not being directly user hostile. Google search is worse now than it was 10 years ago. Netflix has ads with a paid subscription, so does YouTube. Windows is absolute joke, more and more we see user hostile software. Incentives aren’t aligned at all. As people who work in software, I get not wanting to do this stuff at home as well. But honestly I’m hoping for a future where a lot of these services can legit be self hosted by technical people for their local communities. Mastodon is doing this really well IMO. Self hosted software is also getting a lot easier to manage, so I’m quite optimistic that things will keep heading this way.
Note, I’ve got all the things you mentioned down to the UPSes setup in my garage, as well as multiple levels of backups. It’s not perfect, but works for me without much time input vs utility it provides. Each to their own.
If your trust is violated, typically the worst that happens is you are fed a couple more relevant ads or your data is used for some commercial purpose that has little to no effect on your life.
Is it really worth going through so much effort to mitigate that risk?
Again, it's a value judgement, so the answer is largely personal. For me, yes. The social license we give these larger companies after all the violated trust doesn't make sense. If your local shop owner/operator that you talked to everyday had the same attitude towards your when you went shopping and exchanged pleasantries with most weeks, people would confront them about their actions, and that shop wouldn't last long. We have created the disconnect for convenience, and tried to ignore the level of control these companies have on our day to day lives if they are so inclined or instructed to change their systems.
Cloud is just someone else's computer. These systems aren't special. Yes they are impressively engineered to deal with the scale they deal with, but when systems are smaller, they can get a lot simpler. I think as an industry we have conflated distributed systems with really hard engineering problems, when it really matter at what level of abstraction the distribution happens when it comes to down stream complexity.
It introduces some pretty important risks of its own though. If you accidentally delete/forget a local private key or lose your primary email domain there is no recourse. It's significantly easier to set up 2FA and account recovery on a third party service
Note that I'm not saying you shouldn't self-host email or anything else. But it's probably more risky for 99% of people compared to just making sure they can recover their accounts.
I have seen much more stories about people losing access to their Gmail because of a comment flagged somewhere else (i.e YouTube) than people losing access to their domains (it is hard to miss all these reminders about renewal and you shouldn't wait until then anyway so that's something under you control).
And good luck getting anyone from Google to solve your problem assuming you get to a human.
Nice article!
It's heartening in the new millennium to see some younger people show awareness of the crippling dependency on big tech.
Way back in the stone ages, before instagram and tic toc, when the internet was new, anyone having a presence on the net was rolling their own.
It's actually only gotten easier, but the corporate candy has gotten exponentially more candyfied, and most people think it's the most straightforward solution to getting a little corner on the net.
Like the fluffy fluffy "cloud", it's just another shrink-wrap of vendor lockin. Hook 'em and gouge 'em, as we used to say.
There are many ways to stake your own little piece of virtual ground. Email is another whole category. It's linked to in the article, but still uses an external service to access port 25. I've found it not too expensive to have a "business" ISP account, that allows connections on port 25 (and others).
Email is much more critical than having a place to blag on, and port 25 access is only the beginning of the "journey". The modern email "reputation" system is a big tech blockade between people and the net, but it can, and should, be overcome by all individuals with the interest in doing so.
Just for reference, take a look at this email system using FreeBSD:
https://www.purplehat.org/?page_id=1450
p.s. That was another place the article could mention a broader scope, there is always the BSDs, not just linux...
I spend quite some years with linux systems, but i am using llms for configurating systems a lot these days. Last week i setup a server for a group of interns. They needed a docker kubernetes setup with some other tooling. I would have spend at least a day or two to set it up normally. Now it took maybe an hour. All the configurations, commands and some issues were solved with help of chatgpt. You still need to know your stuff, but its like having a super tool at hand. Nice.
Similarly, I was reconfiguring my home server and having Claude generate systemd units and timers was very handy. As you said you do need to know the material to fix the few mistakes and know what to ask for. But it can do the busywork of turning "I need this backup job to run once a week" into the .service and .timer file syntax for you to tweak instead of writing it from scratch.
Isn't depending on Claude to administer your systems rather divergent from the theme of "Self-Host and Tech Independence?"
Claude and others are still in the adoption phase so the services are good, and not user hostile as they will be in the extraction phase. Hopefully by then some agreement on how to setup RAG systems for actual human constructed documentation for these systems will be way more accessible, and have good results with much smaller self hosted models. IMO, this is where I think/hope the LLMs value to the average person will land long term. Search, but better at understanding the query. Sadly, they will also been used for a lot of user hostile nonsense as well.
I think it's just a turbo mode for figuring things out. Like posting to a forum and getting an answer immediately, without all those idiots asking you why you even want to do this, how software X is better than what you are using etc.
Obviously you should have enough technical knowledge to do a rough sanity check on the reply, as there's still a chance you get stupid shit out of it, but mostly it's really efficient for getting started with some tooling or programming language you're not familiar with. You can perfectly do without, it just takes longer. Plus You're not dependent on it to keep your stuff running once it's set up.