this post was submitted on 15 Jan 2025
11 points (100.0% liked)

Linux

50411 readers
884 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
AgreeableLandscape@lemmy.ml
nooter692@lemmy.ml
MarcellusDrum@lemmy.ml
cypherpunks@lemmy.ml
cyclohexane@lemmy.ml
d3Xt3r@lemmy.nz
11
Looking for Help with Weird Connectivity Issues (lemmy.dbzer0.com)
submitted 1 month ago by Kraven_the_Hunter@lemmy.dbzer0.com to c/linux@lemmy.ml
22 comments fedilink hide all child comments
 

I'll preface this by saying that these issues are on a Surface Tablet that I've been using to play around with, so I haven't been too diligent in documenting what changes were made when.

I've got a Surface Go 2 tablet with the LTE modem that I installed Linux Mint onto several months ago. When I first made the switch, cellular connectivity seemed very "touch and go" but Wi-Fi had been solid.

At some point in time (roughly 6 months ago), I switched my home network to using Control D for DNS resolution for about 2 months until I decided it wasn't what I wanted and went back to my default setup which is a Unifi UCG Max gateway using the AdGuard public DNS servers coupled with the built-in ad blocking of the Unifi gateway. This feeds to a separate Wi-Fi mesh network in my home.

About a month ago I noticed that I could no longer reach internet locations on my tablet when connected to my home Wi-Fi network, but I could still access other computers on my LAN just fine, so Wi-Fi was working. Cellular connectivity seemed to have stopped working entirely even though I ran the "lte_modem_fix" that is on github and was seeing several bars of connectivity in the status bar.

Even though websites were inaccessible (Firefox gave me an error saying there was no network connection), in my attempt to try anything I found that I could visit the Control D website even though I stopped subscribing months ago.

On a lark I pulled up my Mullvad VPN app which I have an active subscription to and it let me connect to a server. As soon as I did this, ALL internet sites became available.

Next I took the tablet with me away from home, disabled Wi-Fi and activated the cellular network. Again the bars appeared but I couldn't access any sites. I loaded up Mullvad and was able to connect, after which I could reliably connect to all internet sites. Again, cellular connectivity was never 100% but Wi-Fi was.

How do I even begin troubleshooting and fixing this? Needing a VPN isn't the end of the world, but when at home it gets in the way of accessing local computers so I'd like to get to where the tablet works on Wi-Fi or cellular, with and without a VPN active.

top 22 comments
sorted by: hot top controversial new old
[–] jcarax@beehaw.org 2 points 1 month ago* (last edited 1 month ago) (1 children)

Ok, so does the VPN bring it's own DNS? Some VPNs do, so it may explain why everything suddenly works fine when you connect.

When not connected to VPN, are you able to dig or nslookup internet names? Local names? A server timeout will be very different from an nxdomain or an empty SOA, in the response.

Are you able to telnet to a public web server on TCP/443?

One thought I'm having is, maybe at some point you set a static IP on your wifi interface, but screwed up the subnetting.

Have you ever messed with network manager or systemd-resolved internal settings, maybe trying to setup multicast DNS or caching?

[–] Kraven_the_Hunter@lemmy.dbzer0.com 0 points 1 month ago (1 children)

Yes I believe that Mullvad routes you to their DNS server so that explains why it works when connected to VPN. If I attempt an nslookup when NOT connected to VPN it fails and the server it attempts to contact is 127.0.0.53. When I connect to VPN the nslookup succeeds, and it uses the same server address.

I then disconnect from VPN and ping the ip address that I just looked up (I chose etsy) and the ping goes through so this seems to be a DNS lookup issue. Is 127.0.0.53 the right server address? I would expect it to use my DHCP server address of 192.168.x.x format.

[–] jcarax@beehaw.org 2 points 1 month ago (1 children)

Ok, so something setup 127.0.0.53 as your DNS server, and isn't removing it correctly. I think it's safe to say it's Mullvad, since it works using that DNS server IP when connected. Is that IP in your resolv.conf, or is resolv.conf maybe a stub, and you're using systemd-resolved?

[–] krakenfury@lemmy.sdf.org 2 points 1 month ago (1 children)

127.0.0.53 is the local stub used by systemd-resolved, so OP should pull this thread and comb the docs. If systemd-resolved is installed and not being used, it will cause conflicts with openresolv (most likely alternative).

[–] jcarax@beehaw.org 2 points 1 month ago* (last edited 1 month ago) (1 children)

Interesting that it works when the VPN is connected, though. I also believe that systemd-resolved is installed on just about any system using systemd, but often isn't enabled, without problem. Enabling it would generally involve a resolv.conf symlink and a config, so maybe that config was hijacked by Mullvad (or OP configuring Mullvad), and there's no upstream DNS server available when the VPN isn't connected.

I missed that it's Linux Mint in the original post, and it looks like Mint has started using systemd-resolved. The Arch wiki might be useful to OP on how things are configured:

https://wiki.archlinux.org/title/Systemd-resolved

[–] krakenfury@lemmy.sdf.org 2 points 1 month ago (1 children)

I'm reaching here because I don't know the first thing about Mullvad, but it probably has some script that takes care of it's own DNS needs. I remember the before times, when you had to write up and down scripts that would update resolve.conf directly, then configured OpenVPN to run them on connecting/disconnecting.

It's possible it could be a box checked or config option in Mullvad that broke it by not fixing DNS on it's way down?

OP also said they don't fully remember what was done, so they may have disabled systemd-resolved or installed openresolv or who knows what else.

Fortunately, in this case, they should be able to follow the systemd-resolved docs from the beginning to end up with it working.

[–] jcarax@beehaw.org 1 points 1 month ago (1 children)

Agreed, though I don't think they disabled systemd-resolved, because it still works using 127.0.0.53 when they're connected to the VPN. So the daemon must be running, unless Mullvad itself has a DNS forwarder using the same loopback. I suspect they either hard coded some upstream DNS server for Mullvad, because Mullvad might not have supported systemd-resolved yet. Or maybe they set a permission on the configs, and something changed with the user context of Mullvad processes.

[–] Kraven_the_Hunter@lemmy.dbzer0.com 2 points 1 month ago (2 children)

Thanks for the info, I'll read through the docs and hopefully get this up and running again in the near future. Fortunately, nothing here is mission critical and I can still use the machine with VPN active. Getting resolv.conf back in working order appears to be the right solution.

[–] jcarax@beehaw.org 1 points 2 weeks ago (1 children)

Did you ever come back to this and figure it out? My curiosity is killing me :)

[–] Kraven_the_Hunter@lemmy.dbzer0.com 1 points 2 weeks ago (1 children)

No, unfortunately. It consistently works when connected to Mullvad on both wireless and cellular networks. It briefly worked without Mullvad after a routine software update but its back to its old ways.

What I should probably do next, just to be thorough, is boot Mint or Tails from USB and see if it all works.

Since I dont keep anything on this device, reinstalling wont be a major pain. Ive just had enough other things going on that this hasn't bubbled to the top of my list.

[–] jcarax@beehaw.org 1 points 2 weeks ago (2 children)

If you wanted to, you could post your /etc/resolv.conf and /etc/systemd/resolved.conf here. I don't know if there might be a configuration directory option for systemd-resolved, so keep an eye out for a potential directory like /etc/systemd/resolved.d that might have the configs instead.

[–] Kraven_the_Hunter@lemmy.dbzer0.com 2 points 1 week ago

I went into the Mullvad settings a bit deeper to see why my Surface might be using wireguard tunnels while my desktop doesn't. I didn't see anything related to that, but I did notice that Mullvad has a "Lockdown Mode" which requires you to be connected to Mullvad in order to access the internet. I don't have that active, but I wonder if it is in that mode anyway. I did a quick enable/disable of it to no avail.

[–] Kraven_the_Hunter@lemmy.dbzer0.com 2 points 1 week ago (1 children)

I took a bit to reply because I wanted to wait for the next update to Mullvad. I just installed it this morning and even though I haven't re-started the program, my network connection is dafaulting to go through the Mullvad wireguard servers which is letting everything work. I'm not sure why I have so many copies of the same wg0-mullvad server in my list so that seems suspicious.

Here are the resolv.conf and systemd/resolved.conf files.. really nothing unique other than calling back to 127.0.0.53 for the nameserver like I showed before. My desktop has the same settings and nameserver though. The only difference is that Mullvad on my desktop is not using wireguard servers, so maybe that is causing the issue on my Surface?

resolv.conf

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search .

systemd/resolved.conf

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file (or a copy of it placed in
# /etc/ if the original file is shipped in /usr/), or by creating "drop-ins" in
# the /etc/systemd/resolved.conf.d/ directory. The latter is generally
# recommended. Defaults can be restored by simply deleting the main
# configuration file and all drop-ins located in /etc/.
#
# Use 'systemd-analyze cat-config systemd/resolved.conf' to display the full config.
#
# See resolved.conf(5) for details.

[Resolve]
# Some examples of DNS servers which may be used for DNS= and FallbackDNS=:
# Cloudflare: 1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::10
01#cloudflare-dns.com
# Google:     8.8.8.8#dns.google 8.8.4.4#dns.google 2001:4860:4860::8888#dns.google 2001:4860:4860::8844#dns.google
# Quad9:      9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
#DNS=
#FallbackDNS=
#Domains=
#DNSSEC=no
#DNSOverTLS=no
#MulticastDNS=no
#LLMNR=no
#Cache=no-negative
#CacheFromLocalhost=no
#DNSStubListener=yes
#DNSStubListenerExtra=
#ReadEtcHosts=yes
#ResolveUnicastSingleLabel=no
#StaleRetentionSec=0

[–] jcarax@beehaw.org 1 points 1 week ago

Ok, so the resolv.conf is being used to put systemd-resolved in the forwarding path, with it listening on 127.0.0.53. That's how Mint does things, so don't touch that file.

Your resolved.conf has no DNS servers or fallback DNS servers configured, so it should just use the DNS servers handed out by DHCP. Either your DHCP servers isn't handing out a DNS server (unlikely, since other machines work), NetworkManager was configured to not use DHCP DNS servers, or you're hitting some bug causing the same. I suspect you may have configured NetworkManager for this, maybe it was overriding the VPN DNS. Or maybe you accidentally set the NetworkManager DNS backend to dnsmasq, when it should be systemd-resolved in Mint.

You could try uncommenting that FallbackDNS line and adding a couple space separated DNS servers, maybe your router IP. Mine looks like this:

#DNS= FallbackDNS=1.1.1.1 1.0.0.1 #Domains=

That will hopefully allow VPN DNS to work when it's connected, and fall back to other DNS servers when not. If not, we could try taking a look at NetworkManager configs. It's been a bit, I use systemd-networkd now, but I could spin up a VM.

[–] jcarax@beehaw.org 1 points 1 month ago

And I think it's probably not in resolv.conf, that's a stub that kind of redirects things to systemd-resolved. So I think it's in the forwarder config of that.

Be careful, I was just looking over the Arch docs I linked you to, and I think the configs have changed substantially in the last few months. There's a good chance that the configs in Mint look substantially different.

[–] gomp@lemmy.ml 1 points 1 month ago (1 children)

Assuming you are using networkmanager, the first thing would be to check the DNS settings on your home wifi connection (assuming you are using Gnome, it should be inside "Settings" and then "Network" - sorry if that's wrong, I don't use Gnome).

If you can't locate the setting to change, you can try deleting the whole connection and connecting again (as you would to a new wifi network).

[–] Kraven_the_Hunter@lemmy.dbzer0.com 1 points 1 month ago

I checked and everything was still set to the defaults, which makes sense as I didn't change anything. Regardless, I deleted the connection and re-made it with the same results. No network connectivity outside of my LAN unless I have a VPN active.

Settings are:

Security: WPA/WPA 2 Personal IPV4 using DHCP DNS: Automatic Routes: Automatic IPV6: Active (but unused) with DNS=Auto and Routes=Auto

I did also try resetting the network using the Reset button to no avail.

I may just end up reinstalling the OS if I can't figure anything out.

[–] nyan@sh.itjust.works 1 points 1 month ago (1 children)

Test the network from the lowest level if you haven't already, using ping and the IPv4 address of a common server (for instance, ping 8.8.8.8) to bypass DNS.

If it works, your DNS is borked.

If it doesn't, then there's something more fundamentally wrong with your network configuration—I'd guess it was an issue with the gateway IP address, which would mean it can't figure out how to get to the wider Internet, although it seems super-weird to have that happening with DHCP in the mix. Maybe you left some vestiges of your old configuration behind in a file that your admin GUI doesn't clean up and it's overriding DHCP, I don't know.

[–] Kraven_the_Hunter@lemmy.dbzer0.com 1 points 1 month ago

Thanks for the tip. If I bypass DNS it does appear to work so that's likely the problem. I need to figure out why now and I think it has something to do with a local DNS override of some sort.

[–] data1701d@startrek.website 1 points 1 month ago (1 children)

I feel like I had a problem very much like this with Debian Testing on my Surface Go 1 (and I think my desktop too) a couple years back, and it turned out there was issues with /etc/nsswitch.conf. I can't remember exactly what I did, but this is the current contents of that file:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         files systemd
group:          files systemd
shadow:         files
gshadow:        files

hosts:          files mdns4_minimal [NOTFOUND=RETURN] dns myhostname
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

Compare yours - maybe even post it so I can try to reproduce the issue on my machine. Anyhow, hope it helps, and good luck.

[–] Kraven_the_Hunter@lemmy.dbzer0.com 1 points 1 month ago (1 children)

My nsswitch.conf file looks identical to yours, so nothing to edit there.

I also looked at my resolv.conf and systemd\resolved.conf files.

resolv.conf is a symlink, but is the only file with anything un-commented in the file:

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(
8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search .
[–] data1701d@startrek.website 1 points 1 month ago

Well, it was worth a shot.