IRC chat logs for #ltsp on irc.libera.chat (webchat)


Channel log from 24 October 2021   (all times are UTC)

07:34woernie has joined IRC (woernie!~werner@p5b296c2c.dip0.t-ipconnect.de)
08:40Vercas has left IRC (Vercas!~Vercas@gateway/tor-sasl/vercas, Remote host closed the connection)
08:41Vercas has joined IRC (Vercas!~Vercas@gateway/tor-sasl/vercas)
10:37Vercas has left IRC (Vercas!~Vercas@gateway/tor-sasl/vercas, Remote host closed the connection)
10:37ricotz has joined IRC (ricotz!~ricotz@ubuntu/member/ricotz)
10:37Vercas has joined IRC (Vercas!~Vercas@gateway/tor-sasl/vercas)
12:05woernie has left IRC (woernie!~werner@p5b296c2c.dip0.t-ipconnect.de, Remote host closed the connection)
12:54woernie has joined IRC (woernie!~werner@p5b296c2c.dip0.t-ipconnect.de)
13:16Vercas has left IRC (Vercas!~Vercas@gateway/tor-sasl/vercas, Remote host closed the connection)
13:16Vercas has joined IRC (Vercas!~Vercas@gateway/tor-sasl/vercas)
13:55shored has joined IRC (shored!~shored@user/shored)
14:14eu^ip-74-84-236- has joined IRC (eu^ip-74-84-236-!~eu^ip-74-@74.84.236.83)
14:14eu^ip-74-84-236- is now known as Monkberry
14:16
<Monkberry>
Hey Alkis are you around?
14:18
I ran some updates and some stuff broke. I was hoping you could help.
14:20
<alkisg>
Hi Monkberry
14:20
<Monkberry>
Hi
14:20
<alkisg>
What broke?
14:21
<Monkberry>
Epoptes apparently. I've seen some posts of your about issues with it but none of them seem to fit
14:21
<alkisg>
What happens, e,g, it won't start, the clients don't appear, etc?
14:21
<Monkberry>
Auto login appears to have broke as well for clients on the LTSP platform
14:22shored has left IRC (shored!~shored@user/shored, Quit: ZNC 1.8.2+deb2 - https://znc.in)
14:22
<Monkberry>
I can't look at them or control them as well as not start some of them, all of that worked before the updates
14:23
<alkisg>
Do you want me to check over VNC? I'll need vnc from both your server and a client
14:23
<Monkberry>
On ones that I can start, I can access a root terminal on them via epoptes or I can ssh into them directly
14:23
<alkisg>
(or a VM, if you have one, I don't recall)
14:23
x11vnc -connect alkisg.ltsp.org
14:28
Monkberry: you have two failed services, unifi and ua-messaging
14:28
unifi seems to delay ltsp client startup enough so that lightdm doesn't autologin
14:28
<Monkberry>
I've been watching
14:28
<alkisg>
Let's start by fixing them on the server first
14:28
<Monkberry>
ok
14:28
<alkisg>
Do you know why it fails? Do you need it, should I fix or remove it?
14:29
I'm guessing it's some kind of service for unifi routers?
14:30
<Monkberry>
unifi doesn't even need to run on that box
14:31
You could just remove it, I forgot all about that package
14:31
<alkisg>
OK removing
14:32
<Monkberry>
ok
14:32
<alkisg>
mongodb claims that it's not used by anything else; removing
14:35
I doubt canonical gives linuxmint support via ubuntu-advantage
14:35
Shall I remove that as well?
14:36
Nah many things depend on it; I just masked it
14:37
How easy is it to reboot the server, just to check that it has a clean systemctl status?
14:37
If it's a big deal, we can skip it
14:40
<Monkberry>
We can reboot it
14:42
I'm not there, I'm about an hour away but we can reboot it when that image rebuilds. I also wanted to check and see if I could remotely boot some of the other clients with epoptes after
14:42
I saw you remove a bunch of kernels and stuff. Was all this down to that unifi package?
14:44
<alkisg>
No, these were packages that you have removed but not purged
14:44
So I just removed their "state", nothing important
14:45
If the clients have been shut down without the "wol state" in the NIC, they don't listen for wol anymore
14:45
Lets check if autologin works first
14:45
<Monkberry>
i know but they were (some that I know of) working yesterday before the update
14:45
ok
14:46
should we run "ltsp initrd"?
14:46
before the reboot that is
14:47
I see that you rebooted the ltsp client, I thought you meant the server
14:51
<alkisg>
Let's make the client autologin first
14:55
Monkberry: so, let's assume that unifi started having troubles and is causing systemd to go into a failed state, and the "network-online" target to not be reached
14:55
That means that epoptes-client won't launch, and the NIC won't be put into the "wol mode"
14:55
That then means that even if we fixed it now, you still need to manually poweron the clients
14:55
<Monkberry>
Would that explain why lightdm is not starting?
14:56
<alkisg>
one time only; then they'll get the wol state
14:56
No
14:56
<Monkberry>
I see that the auto login works after a manual reboot of the lightdm service
14:57
ah wait a minute now
14:57
<alkisg>
Did you somehow disable lightdm?
14:57
<Monkberry>
I did manually disable lightdm on the server because someone accidentally logged into it and shut it down
14:57
<alkisg>
Ah, then the clients would also inherit that
14:58
<Monkberry>
So I disabled lightdm on the server so they couldn't login to it anymore
14:58
ah
14:58
<alkisg>
OK, we'll do it like this:
14:58
<Monkberry>
now I'm feeling pretty dumb
14:58
<alkisg>
the server will reach console.target, not multi-user.target
14:58
While lightdm will be enabled
14:58
<Monkberry>
ah perfect
14:58
<alkisg>
*sorry,I meant graphical vs multi-user, I'll google it :D
14:58
OK doing so
14:59
Hmm, how did you disable lightdm?
15:00
<Monkberry>
The server sits in a room where they take the kids if they are not feeling well. etc. Last week one of the teachers went in there, saw the login screen and logged in and then turned it completely off thinking it was just another workstation
15:00
systemctl disable lightdm
15:01
<alkisg>
And yet `systemctl enable lightdm` shows some errors... hmmm...
15:01
<Monkberry>
So me thinking about an easy fix, running that command above just booted the server to a prompt, where if I was there and wanted a gui, I just logged in and ran startx
15:02
<alkisg>
How do you login to the server remotely, e.g. via xrdp, ssh + vnc?
15:02
<Monkberry>
idk maybe the package got removed or borked in some way
15:02
If I'm not there, I use x2go via ssh
15:03
<alkisg>
How are you connected now? I don't see x2go running
15:03
<Monkberry>
via x2go
15:04
<alkisg>
Sorry I misttyped x2g0
15:04
zero instead of oh :)
15:04
<Monkberry>
I'm in Lily Dale, about an hour away (on crappy dsl connection but it's working amazingly fast)
15:04
ah
15:06
For the lightdm thing, an "apt-get install --reinstall lightdm" might have fixed it
15:06
This is too cool, I'm in Lily Dale, you're in Greece and I'm watching you work on a server in Kenmore
15:07
<alkisg>
Haha, and actually I"m at home, remoting to my office PC where the reverse vnc connection happens
15:08
<Monkberry>
lol
15:09
You should probably select /dev/sda with the space bar
15:12
<alkisg>
Yeah I guess it was cloned from another disk
15:12
So... systemctl enable lightdm didn't do it, so as you said I reinstalled it,
15:12
I set the server's default target to multi-user, which does start x2go,
15:12
<Monkberry>
I saw that, yes I clone disks all the time, to save time
15:12
<alkisg>
(but not lightdm),
15:12
while the clients remain with target = graphical, which starts lightdm
15:13
So when you want to "startx" on the server, do this instead:
15:13
sudo systemctl isolate graphical.target.
15:13
Without the dot: sudo systemctl isolate graphical.target
15:13
<Monkberry>
Will I need to do something different to use x2go?
15:14
<alkisg>
Theoretically no
15:14
It should be the same
15:14
<Monkberry>
ok cool
15:16
<alkisg>
Do you have anyone remotely to press the poweron button at the clients, just one time?
15:17
<Monkberry>
we could try with first floor room 104 or room 105
15:18
If need be, I could get someone over there
15:20
<alkisg>
I think the problem is fixed, but someone will need to boot them one time, and then WOL will work after that
15:20
Now the network cards are sleeping without listening to WOLs
15:21
Because lightdm or unifi blocked epoptes-client from running and from enabling their WOLs
15:21
We can poweroff the TV and test if we can wake it up after that :)
15:21
<Monkberry>
I will have someone there in 10 min
15:22
I noticed before that I could remotely start pcs that were on the LTSP system but not others
15:23
<alkisg>
If you install epoptes-client in the others, you'll be able to wol them too
15:23
<Monkberry>
I just tried booting another off the LTSP server
15:23
<alkisg>
Or if you go to network-manager properties and enable wol in the connection settings
15:23
<Monkberry>
I did and it used to work
15:24
The server did shut down last night around 8pm for no apparent reason as well
15:26
The client pc we have been working on should be no difference to any of the others there, except for the distinction that it is an LTSP client
15:27
<alkisg>
I'm not sure I got what you're saying; do you think there's still a problem that we haven't solved?
15:27
<Monkberry>
not sure
15:28
<alkisg>
If so, have someone boot a PC that is affected, so that we can look at its WOL settings
15:28
<Monkberry>
They should be there in 5 min
15:28
<alkisg>
ethtool eth0 => Wake-on: g
15:28
<Monkberry>
I told him we need to boot the pcs in room 104 and 105
15:28
<alkisg>
While if it's like this, it's disabled: Wake-on: d
15:29
It would be better to boot them all, then power them off via epoptes
15:29
<Monkberry>
Yes and they were all working before, except that they might have not gotten shut down correctly or that the server went down before they did
15:29
<alkisg>
Then they'll be listening for wol events, and you'll be able to wake them up at anytime
15:30
<Monkberry>
i know, my guess is that they went down AFTER the server went down, or didn't go down at all.
15:30
and they're now just hanging there
15:30
<alkisg>
If you reboot the ltsp server while the ltsp clients are running, they do reconnect in epoptes
15:31
Also, wol is set at boot, so it wouldn't matter if the ltsp server went down while they were up
15:31
<Monkberry>
interesting
15:31
I've never seen that happen, usually they just hang with some garble on the screen
15:32
<alkisg>
In the old ltsp, they hanged
15:32
In the new ltsp, /home hangs (so the user session hangs), while / continues
15:32
But you have nfs, so /home shouldn't hang either
15:32
<Monkberry>
ah that makes sense
15:32
<alkisg>
It should just continue where it left off
15:33
<Monkberry>
if /home hangs. it would appear the whole thing hangs
15:33
<alkisg>
nbd and sshfs hang, nfs continues
15:33
<Monkberry>
the homes are nfs mounted
15:33
<alkisg>
Old ltsp = nbd and sshfs; new ltsp = nfs and sshfs; your ltsp = nfs and nfs
15:33
So nothing should hang
15:33
<Monkberry>
well the whole thing really
15:34
<alkisg>
We can test if you want, we can reboot the server while TV is up
15:34
<Monkberry>
that nbd was a NIGHTMARE
15:34
ok
15:34
<alkisg>
You do have a static ip on the server, right?
15:34
<Monkberry>
yes
15:34
<alkisg>
OK, if you want, reboot the server
15:34
<Monkberry>
ok
15:34
<alkisg>
Hopefully we'll be able to reconnect via x2go etc :)
15:35
<Monkberry>
ah yes
15:35
that wouldn't be good if I couldn't get in somehow
15:37
Did you install grub to /dev/sda?
15:37
ah ok hang on
15:38
<alkisg>
Yes, although it shouldn't matter it was already there
15:39
See, everything is perfect :D
15:39
nfs ==> real file system! :D
15:39
<Monkberry>
The camera viewer (mvp) was running and then it stopped
15:41
<alkisg>
Maybe it dies if it can't write to disk for a minute
15:41
That's an app issue though
15:41
<Monkberry>
I just tried to use epoptes to control the pc and I can't login
15:41
try it
15:42
oh now it worked
15:44
it seems to work (sometimes) or with a very long delay
15:45
<alkisg>
I think that's a bug in the new epoptes code
15:45
I haven't pinpointed it, let me have a look
15:45
<Monkberry>
xtigervncviewer?
15:46
<alkisg>
That part is ok; what is not ok is that it's trying to run [xtigervncviewer,
15:47
With [ and comma
15:47
The weird thing is that it does this SOME times only, so I haven't been able yet to see what goes wrong
15:49
<Monkberry>
maybe a clue, I just had someone over there and manually started the pcs in 104 and 105 and we see nothing on epoptes
15:49
I could ssh into those if needed
15:49
<alkisg>
Sure yes please
15:49
<Monkberry>
ok
15:49
<alkisg>
I don't think it's related to epoptes though
15:54
I don't see any dhcp requests etc after the ltsp server has been booted
15:54
Let's reboot the TV pc to check if it boots
15:56
<Monkberry>
ok, I'm logged into both pcs now, 104 and 105
15:56
<alkisg>
$ epoptes-client
15:56
Epoptes-client connecting to server:789... ...done
15:56
epoptes-client: 382: [xtigervncviewer,: not found
15:56
<Monkberry>
hang on and I'll give you shells on them
15:56
<alkisg>
Thanks
16:04
Monkberry: there's some firewall involved somewhere
16:04
<Monkberry>
Should not be anything on the inside
16:04
I can reboot the firewall if you like
16:05
it's pfsense and it is the dhcp server as well
16:08
<alkisg>
Hmm no
16:08
The problem is somehow related to the previous epoptes certificate
16:08
By generating a new one, the problem is solved,
16:09
but then, you'll need to run `epoptes-client -c` in all your clients...
16:09
Is that acceptable?
16:09
<Monkberry>
room 104 is a stand alone install (not ltsp) but it does mount it's home via nfs
16:09
<alkisg>
You'll need to ssh to everyone of them, and run epoptes-client -c
16:09
Then epoptes-client will work
16:10
<Monkberry>
I can do that
16:10
<alkisg>
If that's OK, we can stop debugging, problem solved
16:10
OK, can you also ssh to the TV ltsp client?
16:10
<Monkberry>
I do appreciate all your help and I'll take care of you
16:10
<alkisg>
Because that one now doesn't have the correct certificate
16:10
<Monkberry>
sure hang on
16:11
<alkisg>
There's still the problem with the [xtighervncviewer', but that one's on me :)
16:11
I'll try to find out when it happens
16:11
<Monkberry>
That TV pc DOES boot via epoptes though, that must be because it is on LTSP?
16:12
I have noticed that (at least it used to be), that the wol needed to be initiated from .11 if the PC was on .11 (old vlan stuff)
16:12
however, unless the pc is on the .10 side, it would NOT boot into the LTSP server (again, old vlan stuff)
16:12
<alkisg>
That part is because you can't relay WOLs easily
16:16
What's the IP for .105?
16:16
*Room 105
16:17
<Monkberry>
172.16.10.61 I believe or .11.61
16:17
<alkisg>
OK ssh there so that we fix that one as well
16:17
It's just an `epoptes-client -c; systemctl restart epoptes`
16:18
<Monkberry>
no it's 172.16.11.11
16:19
<alkisg>
OK that's it. Btw you can use the hostname.local instead of the ip.
16:19
Actually give me a few minutes to check the tigervnc issue
16:19
<Monkberry>
ok I can fix the rest up now, thank you very much! What did the issue appear to be? Multiple I know
16:19
ok
16:20
<alkisg>
The main issue was that you disabled lightdm
16:20
The epoptes certificate issue, I've no idea what caused it, e.g. expiry or certificate chains or cipher mismatch
16:21
<Monkberry>
ah
16:21
<alkisg>
While this new xtigervnc issue is on me, it started with the latest patches a couple of weeks ago
16:21
From what I can see, it fails one time, it succeeds the next; very weird
16:21
<Monkberry>
one of those perfect storm scenarios
16:22
<alkisg>
Btw realvnc-vnc-viewer is a lot better than tigervnc and supports automatic scaling, if you want to switch to that
16:22
<Monkberry>
Yes, I would love to
16:23
<alkisg>
(x2go somehow freezes for a while when vnc'ing to the tv pc)
16:23
<Monkberry>
On the clients before, I could ctrl-enter and it would unmaximize (usually their windows vm) and I could then access their linux desktop
16:23
or ctrl-alt-enter
16:23
<alkisg>
Can you still operate the screen via x2go?
16:24
It's frozen here
16:24
<Monkberry>
yes, it looks like you got kicked out
16:24
<alkisg>
x11vnc -connect alkisg.ltsp.org
16:24
<Monkberry>
I'll vnc you again
16:31
<alkisg>
Monkberry: please close the tv-pc VNC window
16:31
It freezed my vnc again :D
16:31
Thanks
16:32
Close it again please :)
16:35
: -listen, 52217] 52217
16:35
I think python somehow fails to use the array properly half of the times :(
16:38
Monkberry: please close it again
16:38
OK ready I found it
16:38
It's partly my fault, partly python's
16:38
I'm using the same variable name, cmd, in two scopes
16:39
And python sometimes sees the wrong scope
16:39
I'll just rename the var
16:39
<Monkberry>
ah very good
16:39
<alkisg>
It's fixed now for you, but I'll also send an update for all the other users
16:39
We can close VNC now, good work for both of us! :)
16:40
<Monkberry>
Thank you Alkisg! Do I still have to ssh to all the clients and rerun epoptes-client -c?
16:40
<alkisg>
Yes
16:40
(except for ltsp clients of course)
16:40
<Monkberry>
ok no biggie
16:40
<alkisg>
We solved 3 separate issues today!
16:40
<Monkberry>
did you change the vnc server?
16:41
Is it back to where I can ctrl-alt-enter?
16:42
<alkisg>
Ah no I forgot about that
16:44
Monkberry: my kid needs me for something, and you have many pcs; let's do this instead:
16:44
Please read our new documentation about vnc viewers: https://epoptes.org/documentation/vnc/
16:45
Try to apply it yourself (the realvnc-vnc-viewer is the best; the xvnc4viewer the second-best; the ssvnc the third-best)
16:45
And if you can't make it work for some reason, I can help you another time
16:46
<Monkberry>
ok sounds good Alkisg! Thank you again and I'll paypal ya.
16:46
<alkisg>
Thank you too! :)
16:47
<Monkberry>
Now, if I can figure out how to print this conversation out.....
16:55
I figured it out, have a good one!
16:55Monkberry has left IRC (Monkberry!~eu^ip-74-@74.84.236.83, Quit: Client closed)
18:31vagrantc has joined IRC (vagrantc!~vagrant@2600:3c01:e000:21:21:21:0:100b)
18:53woernie has left IRC (woernie!~werner@p5b296c2c.dip0.t-ipconnect.de, Remote host closed the connection)
21:35ricotz has left IRC (ricotz!~ricotz@ubuntu/member/ricotz, Quit: Leaving)
23:25vagrantc has left IRC (vagrantc!~vagrant@2600:3c01:e000:21:21:21:0:100b, Quit: leaving)
23:56lucas__ has joined IRC (lucas__!~lucascast@189.90.44.253.jupiter.com.br)
23:59lucas_ has left IRC (lucas_!~lucascast@177-185-128-123.static.isotelco.net.br, Ping timeout: 264 seconds)