IRC chat logs for #ltsp on irc.libera.chat (webchat)
Channel log from 30 January 2019 (all times are UTC)
| 00:56 | GodFather has left IRC (GodFather!~rcc@wsip-66-210-242-210.ph.ph.cox.net, Ping timeout: 268 seconds) | |
| 03:01 | mads2 has left IRC (mads2!~newmember@2804:14c:878d:9e87:4132:7c76:3e2a:8ed1, Ping timeout: 252 seconds) | |
| 04:44 | mads2 has joined IRC (mads2!~newmember@2804:14c:878d:9567:4132:7c76:3e2a:8ed1) | |
| 05:30 | mads2 has left IRC (mads2!~newmember@2804:14c:878d:9567:4132:7c76:3e2a:8ed1, Ping timeout: 252 seconds) | |
| 06:07 | vagrantc has left IRC (vagrantc!~vagrant@unaffiliated/vagrantc, Quit: leaving) | |
| 07:26 | nehemiah has joined IRC (nehemiah!~nehemiah@hs-user-138.wia.cz) | |
| 07:45 | ricotz has joined IRC (ricotz!~ricotz@ubuntu/member/ricotz) | |
| 10:56 | Faith has joined IRC (Faith!~Paty_@unaffiliated/faith) | |
| 12:33 | vsuojanen has left IRC (vsuojanen!~vsuojanen@cable-hml-585686-205.dhcp.inet.fi, Remote host closed the connection) | |
| 12:38 | mads2 has joined IRC (mads2!~mads2@2804:14c:878d:9567:4958:4af4:ea5f:aff3) | |
| 13:51 | <JuJUBee> in /opt/ltsp/images I see 2 one ends in .old and both same size. I just tried to update-image and ran out of space. Can I remove the .old now without any ill effects?
| |
| 13:59 | <alkisg> Yes, see the man page
| |
| 14:00 | It's an automatic backup, revertable with -r
| |
| 14:13 | <||cw> but note if any clients haven't been rebooted they'll still have a lock on the inodes and the space will not be freed until they reboot
| |
| 14:59 | spaced0ut has joined IRC (spaced0ut!~spaced0ut@unaffiliated/spaced0ut) | |
| 16:22 | GodFather has joined IRC (GodFather!~rcc@wsip-66-210-242-210.ph.ph.cox.net) | |
| 16:29 | mads2 has left IRC (mads2!~mads2@2804:14c:878d:9567:4958:4af4:ea5f:aff3, Ping timeout: 252 seconds) | |
| 16:37 | <alkisg> Anyone with ipv6 knowledge? (1) Does each client automatically get an ipv6 address, even when there's no dhcp6 server?
| |
| 16:37 | And(2), can the ltsp server get a "static" ipv6 address, maybe related to its mac address, that clients could use to contact it, even when they didn't know its ipv4 address?
| |
| 16:37 | (trying to solve the proxydhcp/ipappend 3 issue, where the clients, in the initramfs, can't contact the server unless its ip is passed in the kernel cmdline)
| |
| 16:41 | <quinox> I have setup IPv6 at my work
| |
| 16:41 | every client gets multiple IPv6 addresses even without a DHCP router present
| |
| 16:42 | <alkisg> Are they static? E.g. if you move a computer to another ipv6 network, will it have the same address?
| |
| 16:42 | <quinox> DHCP is a bit of a mess, I settled on SLAAC
| |
| 16:43 | <alkisg> I read that it's somehow related to the client mac address, but I didn't yet read if it's completely static, independed of the network id...
| |
| 16:43 | <quinox> with SLAAC every PC figures the last half of the IP address for itself, and the first half is gathered from the DHCP server
| |
| 16:43 | but there are no leases like with IPv4
| |
| 16:43 | <alkisg> ...the actual question is, if I tell the clients the server mac address, will they be able to calculate its ipv6 ip...
| |
| 16:44 | OK, that sounds workable; the client will learn the "network id" from the slaac server, and the "client id" from the server itself...
| |
| 16:45 | Hopefully busybox has enough tools to set that up...
| |
| 16:47 | <quinox> I'm happy to test things out my LAN
| |
| 16:47 | I set up IPv6 many years ago, it has been working great
| |
| 16:49 | <alkisg> Wait no, I think what I remembered is "link-local" addresses, which are derived from mac addresses, not "slaac" addresses...
| |
| 16:49 | * alkisg reads on... | |
| 16:50 | <quinox> IIRC link-local is only between hardware
| |
| 16:50 | the wire between devices
| |
| 16:50 | no arp to map Mac to IP
| |
| 16:51 | you talk to the router using the link-local address
| |
| 16:53 | <alkisg> Ah, so it wouldn't allow client <=> switch <=> server ?
| |
| 16:54 | <quinox> AFAIK no
| |
| 16:54 | <alkisg> Hmm, it works in vbox
| |
| 16:55 | <quinox> link-local addresses start with fe80, ping doesn't even accept an address like that
| |
| 16:57 | you can have multiple global addresses I think
| |
| 16:57 | one for LAN "global" and one for internet "global" maybe?
| |
| 16:58 | <alkisg> $ ping6 -I enp2s0 fe80::5054:ff:fe25:e337
| |
| 16:58 | PING fe80::5054:ff:fe25:e337(fe80::5054:ff:fe25:e337) from fe80::a9ae:ebb8:f1bb:2677%enp2s0 enp2s0: 56 data bytes
| |
| 16:59 | <quinox> I had to learn quite a bit to understand IPv6, it's quite different from IPv4
| |
| 16:59 | <alkisg> I ping the VM client from the server...
| |
| 16:59 | Let me see if the client can wget from the server
| |
| 16:59 | If yes, problem solved :)
| |
| 17:00 | <quinox> you are right, that also works for me when I try to ping between two physical servers
| |
| 17:00 | (between the same router)
| |
| 17:01 | the notation ping shows is valid, you can do 'ping6 fe80::a9ae:ebb8:f1bb:2677%enp2s0' as well
| |
| 17:01 | alright, because it's a link-local address you need to specify the link, otherwise ping doesn't know what to do
| |
| 17:02 | inet6 fe80::21a:a0ff:fe32:d86e prefixlen 64 scopeid 0x20<link>
| |
| 17:02 | ether 00:1a:a0:32:d8:6e txqueuelen 1000 (Ethernet)
| |
| 17:03 | <alkisg> I didn't yet find how to do it in busybox wget
| |
| 17:03 | <quinox> you can see the MAC address as part of the link-local address
| |
| 17:07 | Curl works with the '%iface' notation, wget refuses
| |
| 17:07 | <alkisg> Yeah additionally php -S doesn't listen in ipv6 :D
| |
| 17:08 | <quinox> https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1566930
| |
| 17:09 | GodFather has left IRC (GodFather!~rcc@wsip-66-210-242-210.ph.ph.cox.net, Ping timeout: 272 seconds) | |
| 17:10 | <quinox> if you try '[::0]'?
| |
| 17:12 | works for me
| |
| 17:12 | <alkisg> ...it worked but with a weird :3000 port
| |
| 17:13 | Unfortunately curl isn't available in stock initramfs, let me see if something can be done with wget...
| |
| 17:13 | <quinox> php -S '[::0]:80' <-- works for me as well
| |
| 17:14 | <alkisg> Just got that :D ! thanks
| |
| 17:14 | <quinox> at nearly all places you'll have to wrap the IPv6 address in square brackets
| |
| 17:15 | <alkisg> I wonder if I can put the address e.g. to /etc/hosts, to work around the wget issue
| |
| 17:15 | <quinox> ah, nice trick... perhaps
| |
| 17:16 | (I'm now on wayland, the alt-tab behavior drives me nuts - why group all terminals together under "terminal" ?!)
| |
| 17:18 | <alkisg> I think there's also Alt+`
| |
| 17:18 | To switch within a group
| |
| 17:19 | <mwalters> I use a 68-key compact keyboard
| |
| 17:19 | alt+` is a pain ;)
| |
| 17:19 | <quinox> at first glance it seems adding %enp1s0 renders the /etc/hosts entry invalid
| |
| 17:20 | I can't ping it if that's present, I can (fail to) ping it without
| |
| 17:20 | <alkisg> curl --interface enp2s0 http://alkis6/file
| |
| 17:20 | That works for me
| |
| 17:21 | But wget doesn't have an --interface switch :(
| |
| 17:21 | (without the %enp2s0 in hosts)
| |
| 17:22 | The bug report says: "However, with the wget included in BusyBox v1.25.1, the same command is working! "
| |
| 17:22 | <quinox> https://unix.stackexchange.com/questions/174767/ipv6-zone-id-in-etc-hosts
| |
| 17:22 | <alkisg> Meh. This is what we want, wget busybox... let's see...
| |
| 17:23 | You can test with "busybox wget parms"
| |
| 17:24 | Yey
| |
| 17:24 | Got it
| |
| 17:25 | I guess that even adds a bit of security, if the server is listening only on link-local addresses, it won't be accessible from outside
| |
| 17:26 | busybox wget http://[fe80::a9ae:ebb8:f1bb:2677%enp2s0]/file
| |
| 17:26 | <quinox> the same goes if it's listening on its 192. or 10. address
| |
| 17:27 | <alkisg> Yes but that's not really static, while the ipv6 address can be hardcoded
| |
| 17:29 | So to sum up, we can pass "ltsp.server_ipv6=fe80::a9ae:ebb8:f1bb:2677" in the kernel cmdline, and allow the client to discover the server from the initramfs even in proxy dhcp cases. Great!
| |
| 17:29 | No "IPAPPEND 3" hacks needed anymore :)
| |
| 17:30 | <quinox> isn't that missing an interface?
| |
| 17:31 | <alkisg> The interface name is the client one, the server doesn't know it
| |
| 17:31 | The client will use the one that got an ipv4 address
| |
| 17:32 | <quinox> cool
| |
| 17:32 | <alkisg> Thanks for helping quinox! :)
| |
| 17:32 | <quinox> anytime
| |
| 17:35 | kjackal has left IRC (kjackal!~quassel@2a02:587:3101:f300:18fa:1162:c324:7650, Ping timeout: 252 seconds) | |
| 17:35 | kjackal_v2 has joined IRC (kjackal_v2!~quassel@80.107.182.228) | |
| 17:39 | <alkisg> Eh, 2 `php -S` instances are needed to listen to 2 link-local addresses, but anyway, nbd that listens on 10809 is the same :)
| |
| 17:40 | <mwalters> to be honest
| |
| 17:40 | sounds like a good time to apt remove php
| |
| 17:40 | ;)
| |
| 17:40 | <alkisg> Haha, why?
| |
| 17:40 | python -m SimpleHTTPServer => fails lots of times
| |
| 17:40 | php -S => I haven't seen it fail
| |
| 17:41 | <mwalters> require('http').createServer((req, res) => res.end('hello')).listen(80)
| |
| 17:41 | there you go ;)
| |
| 17:41 | <alkisg> What's that, ruby?
| |
| 17:41 | <mwalters> javascript/node
| |
| 17:41 | <alkisg> Yeah sure you go tell teachers how to install that from npm :P
| |
| 17:41 | <mwalters> a really convoluted one liner... that actually works ;)
| |
| 17:42 | fair enough... "php, at least it's not perl" ;)
| |
| 17:43 | <alkisg> If we wanted to help ltsp contributors... what would they prefer, php, python or nodejs/javascript, for server side scripting?
| |
| 17:43 | <mwalters> I just have a terrible adversion to php... I've maintained waaaay too many legacy php apps
| |
| 17:43 | what's the usecase?
| |
| 17:43 | I think python is popular in higher ed
| |
| 17:43 | at least within the scientific community
| |
| 17:43 | nodejs/javascript is usually webapp hipsters
| |
| 17:44 | <alkisg> Let's say that lts.conf gets transformed to https://server/config?phase=boot
| |
| 17:44 | So, if they want to edit the configuration sent to the client at that phase, they'd put some server-side code
| |
| 17:44 | That would parse some /etc/ltsp/lts.conf, or maybe do load - balancing based on server load, etc, usually simple things
| |
| 17:45 | <mwalters> some sort of markup, yaml? would probably be better for the configs, then use whatever to serve the config to the clients?
| |
| 17:45 | <alkisg> The user editable configs would be .ini or yaml or something, yes, but the code parsing those configs and transforming them into shell scripts for the client, would be... php? pyton?
| |
| 17:45 | I.e. that "whatever" that you said
| |
| 17:46 | Talking about ltsp programmers/contributors, not about users...
| |
| 17:46 | <mwalters> gotcha
| |
| 17:46 | I think it's reasonable to expect either python2.7 or 3 to be on nearly every linux installation
| |
| 17:47 | (and I'm not really a python person)
| |
| 17:47 | <alkisg> I don't trust its internal http server, and if we're going to use e.g. twisted, we might as well use php; i imagine more devs know php rather than twisted?
| |
| 17:47 | Epoptes is using python-twisted, it's fine; but it's a bigger dependency than php
| |
| 17:47 | <mwalters> does django not use the built in http server?
| |
| 17:48 | no, it doesn't, does it
| |
| 17:48 | yeah, I get what you're saying
| |
| 17:48 | I was spoiled by node, which... well, you saw the one liner
| |
| 17:49 | <alkisg> sudo apt install nodejs is simple enough; but I'm worried about how many devs will be able to contribute with server-side javascript code
| |
| 17:49 | Is server-side javascript common nowadays?
| |
| 17:49 | <mwalters> node has terrible disk io
| |
| 17:50 | fairly
| |
| 17:50 | if you're only reading a couple files, it's probably OK
| |
| 17:50 | Node is/was pretty huge
| |
| 17:50 | I think netflix and airbnb use it extensively
| |
| 17:51 | A lot developers (myself included) switched to typescript because we finally saw the light
| |
| 17:51 | Then we got fed up with the convoluted stack and switched to go or rust ;)
| |
| 17:54 | It would be simple enough to set up http handlers w/ node, though
| |
| 17:54 | most of it is preference
| |
| 17:55 | you could use a compiled language like go and just distribute binaries
| |
| 17:55 | no runtime needed
| |
| 17:56 | go looks similar to C
| |
| 17:56 | some folks have issues with how it does error handling (i.e., it doesn't, do it yourself)
| |
| 17:58 | <alkisg> (07:51:07 μμ) mwalters: A lot developers (myself included) switched to typescript because we finally saw the light => yeah I thought people were moving away from that nowadays
| |
| 17:58 | (07:54:11 μμ) mwalters: It would be simple enough to set up http handlers w/ node, though => can it be done with `apt install`, without using `npm install` i.e. without downloading things under /home/username?
| |
| 17:58 | (07:55:24 μμ) mwalters: you could use a compiled language like go and just distribute binaries => nah I really prefer interpreted languages for ltsp, makes testing much easier
| |
| 17:59 | I read a couple of articles on python vs php vs nodejs... I think all those would more or less fit...
| |
| 18:00 | <mwalters> you can install node without npm, installing node will install npm, though
| |
| 18:00 | vagrantc has joined IRC (vagrantc!~vagrant@unaffiliated/vagrantc) | |
| 18:01 | <mwalters> re: testing... the go debugger isn't bad, can't speak to rust, though
| |
| 18:02 | <alkisg> Oh I mean, "let's change that `sed /a/b/` to `sed /a/c/` and see if it works... oh no I need to build it again, saving isn't enough...
| |
| 18:02 | <mwalters> fair enough. Go compiles fairly quickly, though, even on lower end systems
| |
| 18:02 | <alkisg> Compilation usually isn't enough, there's an installation step involved too
| |
| 18:03 | <mwalters> to distribute, yes
| |
| 18:03 | <alkisg> While with interpreted languages, testing can be done by editing directly /usr/share/script
| |
| 18:03 | <mwalters> fair
| |
| 18:03 | looks like nodejs 8.10 is the version published for 18.04 right now
| |
| 18:05 | And yeah, node performance is probably fine for what I think you're trying to accomplish here... when I mentioned bad disk IO, my usecase was loading up like... 20 csv files with a million lines each and parsing them ;)
| |
| 18:07 | I was working on a logistics management system for a frieght company a couple years back, building up a system to analyze toll road fees against our route planning software to make sure drivers weren't abusing our toll transponder devices ;)
| |
| 18:17 | <alkisg> vagrantc: what would you vote? If lts.conf was going to be replaced with https://server/config?phase=boot, would you prefer php, python or node.js for the server side scripting?
| |
| 18:18 | <vagrantc> alkisg: personally, python
| |
| 18:19 | alkisg: node moves very quickly and i suspect it will be hard to maintain compatibility across multiple releases
| |
| 18:19 | <mwalters> from personal experience, it really isn't
| |
| 18:20 | <vagrantc> and php shouldn't be used for new projects, in my personal bias
| |
| 18:20 | <mwalters> node 8.10, which is what ships with ubuntu 18.04 is an LTS release and will be maintained until april 2020
| |
| 18:20 | eer, october 2020
| |
| 18:21 | also, agree for the same reasons on php
| |
| 18:21 | I really think python is the way to go, though... simply because it's included out of the box in basically every distribution
| |
| 18:21 | even if SimpleHTTPServer isn't the way to go
| |
| 18:22 | I'm much more comfortable in node than python... but I don't recoil in horror at the thought of using it ;)
| |
| 18:22 | <vagrantc> debian's current stable release has 4.8.2
| |
| 18:22 | for node
| |
| 18:22 | nodejs
| |
| 18:23 | <mwalters> that's ancient
| |
| 18:23 | <vagrantc> granted, debian's about to release a new version
| |
| 18:23 | <mwalters> yeah, node4 has been out of maintenance for... a long time ;)
| |
| 18:23 | <vagrantc> in process of upgrading to 10.15.0
| |
| 18:24 | so, like i said, node is fast-moving :P
| |
| 18:24 | <mwalters> absolutely... I will say, I took an express app from node4 to 6 without having to change a single line of code, though
| |
| 18:25 | <vagrantc> if it can't handle an LTS distro with a ~2 year release cycle ...
| |
| 18:25 | <mwalters> The LTS releases of node will, though
| |
| 18:25 | <vagrantc> let alone a 5 year release cycle
| |
| 18:26 | actually, the debian LTS release has nodejs 0.10.29
| |
| 18:27 | <mwalters> but yeah, python is still probably the way to go
| |
| 18:27 | <vagrantc> but i don't realistically even support the LTSP version in that old release anyways
| |
| 18:27 | <mwalters> or... maybe perl ;)
| |
| 18:27 | just to make sure it's stable ;)
| |
| 18:28 | <vagrantc> if for some reason python wasn't available, sure.
| |
| 18:28 | :)
| |
| 18:40 | <alkisg> Python would be python-twisted, of course; and e.g. it took them years to support python3
| |
| 18:40 | While php has a much more active communite related to web programming
| |
| 18:41 | Python is great, but I'm not so sure it's mature for web programming
| |
| 18:42 | <mwalters> p sure facebook is mostly python
| |
| 18:42 | <vagrantc> there are other frameworks than python-twisted
| |
| 18:42 | <mwalters> django has been around for years
| |
| 18:42 | <alkisg> Well, unless we rely on some web server like apache/nginx/lighttpd, but I think that's too much overhead
| |
| 18:42 | <mwalters> python absoutely is suited to web applications
| |
| 18:42 | oh, yeah... wsgi
| |
| 18:42 | <alkisg> vagrantc: which one would you suggest?
| |
| 18:43 | <vagrantc> django is the first that comes to mind, but it's a bit bare-bones
| |
| 18:43 | <mwalters> django also doesn't include a "production" http server
| |
| 18:43 | at least the last time I used it
| |
| 18:43 | django would be plenty to do what you wanted I think... it comes with a router, controller, and templates... not sure what more you'd need
| |
| 18:44 | <alkisg> Isn't django an overkill for an lts.conf replacement?
| |
| 18:44 | <mwalters> there's also pylons, which is even more bare bonsed
| |
| 18:44 | absolutely
| |
| 18:44 | you just want to serve the config file over http?
| |
| 18:44 | <alkisg> "Calculate" the config file and send it
| |
| 18:44 | Based on server networking params,maybe load, maybe xsessions...
| |
| 18:44 | <mwalters> that'll be based upon some header information in a get request from the client?
| |
| 18:45 | * vagrantc thinks a web server is overkill for an lts.conf replacement | |
| 18:45 | <alkisg> Yes, including some state to keep things safer
| |
| 18:45 | <mwalters> state meaning session management?
| |
| 18:45 | and some sort of authentication?
| |
| 18:46 | <alkisg> No
| |
| 18:46 | Let's say state = a uuid
| |
| 18:46 | You give it to the client at the initramfs stage
| |
| 18:46 | <mwalters> to identify the client?
| |
| 18:46 | <alkisg> And then the client uses it to request additional information
| |
| 18:46 | Yes
| |
| 18:46 | <mwalters> gotcha
| |
| 18:46 | <alkisg> So that you don't send LDM_PASSWORD etc to any client that asks for it
| |
| 18:46 | <mwalters> ;)
| |
| 18:46 | <alkisg> So, only the initramfs initial stage is insecure, the rest are secure
| |
| 18:47 | <mwalters> I mean, php requires a webserver still, doesn't it?
| |
| 18:47 | <alkisg> php -S doens't
| |
| 18:47 | <mwalters> (I haven't done any real php stuff since like... 3/4)
| |
| 18:47 | <alkisg> It looked stable for such simple things here
| |
| 18:47 | Much more stable than python -m SimpleHTTPServer
| |
| 18:48 | <mwalters> yeah, I mean the module docs say as such
| |
| 18:48 | s/such/much
| |
| 18:48 | I get where you're coming from... node would solve the http server issue, the http module is rock solid
| |
| 18:48 | I think "linux people" would prefer python
| |
| 18:49 | and I'm preferential given in the environment... but yeah, the lack of http server is an issue
| |
| 18:49 | I know nothing about `php -S`, but hate the language ;)
| |
| 18:51 | ... HTTP::Server::Simple doesn't have the same big red warnings as python's...
| |
| 19:00 | <quinox> I earn money with Django; it's fantastic, but seriously overkill for LTSP
| |
| 19:00 | <mwalters> ^^^
| |
| 19:01 | it also doesn't solve the http server issue
| |
| 19:01 | <quinox> indeed, the built-in dev server isn't meant for anything more serious than developing
| |
| 19:03 | <mwalters> maybe a python person can answer what "It only implements basic security checks" means: https://docs.python.org/2/library/simplehttpserver.html, because I find it really $^% vague
| |
| 19:05 | <quinox> CSRF?
| |
| 19:06 | kjackal_v2 has left IRC (kjackal_v2!~quassel@80.107.182.228, Ping timeout: 268 seconds) | |
| 19:06 | kjackal has joined IRC (kjackal!~quassel@2a02:587:3101:f300:18fa:1162:c324:7650) | |
| 19:07 | <quinox> it seems to do little, so not that much that could go wrong
| |
| 19:08 | <mwalters> yeah, I can't imagine what "security checks" we couldn't do in the application itself
| |
| 19:08 | and maybe I'm not understanding how this works... but I'm used to creating a webserver and doing all the request and response handling in the code itself, not serving files
| |
| 19:09 | which may be part of my aversion to php these days
| |
| 19:09 | * mwalters shrugs | |
| 19:09 | <alkisg> I'm not sure what that means, php handles the requests with code...
| |
| 19:09 | <mwalters> I think the idea of having a webserver opens up some possibilities for client handling, which is cool
| |
| 19:09 | so it's not just html mixed iwth php anymore?
| |
| 19:09 | (again, last time i used PHP was back in version 2, 3 and 4)
| |
| 19:10 | <alkisg> AFAIK it was always possible to do <?php and write plain php without html
| |
| 19:10 | And, if we're to send shell-sourceable files to the client, we wouldn't want any html anyway
| |
| 19:10 | <mwalters> sure, but it was certainly structured differently than an HTTP response handler in something like node/go/c/whatever
| |
| 19:10 | <alkisg> HTML could come in the future, if someone wanted to implement a frontend for administration, but that's too far away to consider now
| |
| 19:17 | <mwalters> alkisg: do you have any specific concerns surrounding SimpleHTTPServer?
| |
| 19:17 | p sure django uses it for it's development server, I can't say I had issues with it when I used it
| |
| 19:34 | <alkisg> mwalters: ah, for example, a single request from ie11 hangs the server, so that it needs a restart
| |
| 19:35 | Faith has left IRC (Faith!~Paty_@unaffiliated/faith, Quit: Leaving) | |
| 19:37 | <alkisg> Well I guess bugs can be filed as usual... maybe it's not that bad after all
| |
| 19:37 | And I do like the idea of having a "process" instead of a "file", server-side
| |
| 19:38 | <quinox> ugh, IE and Chrome both are good at keeping hidden background connections open -_-
| |
| 19:49 | GodFather has joined IRC (GodFather!~rcc@wsip-66-210-242-210.ph.ph.cox.net) | |
| 19:50 | <mwalters> can't say I've experienced that
| |
| 19:50 | but who uses ie11 anyways? ;)
| |
| 19:52 | danboid has joined IRC (danboid!~dan@cpc126962-macc4-2-0-cust227.1-3.cable.virginm.net) | |
| 19:53 | <alkisg> I was just testing some html5 educational apps I made, to see if they work in ie too
| |
| 19:53 | <danboid> Hi guys! Does anyone know the 'best' way to boot into LTSP from the windows boot manager, ideally one tha supports both BIOS and UEFI machines?
| |
| 19:53 | <alkisg> And it was like, 3 server restarts, one test
| |
| 19:54 | danboid: I'm working on it, it might be ready in the summer :P
| |
| 19:54 | For now, win32-loader for bios
| |
| 19:54 | And later on, grub+ipxe.efi, or grub+kernel
| |
| 19:54 | Or just plain pxe, without bothering with the windows boot manager
| |
| 19:55 | <mwalters> I used to do the opposite I think, I had a pxelinux set up that would defer to the local boot manager
| |
| 19:55 | <danboid> We have to have the Windows boot manager anyway to boot Windows
| |
| 19:55 | <mwalters> this was waay back I think...
| |
| 19:55 | <alkisg> I did that once, but it requires the pxe server to be up in order to boot windows, so I gave up on it
| |
| 19:55 | <mwalters> so I could load dban/WDS/etc waaay back
| |
| 19:55 | <danboid> We currently use grub4dos so its BIOS only
| |
| 19:55 | <alkisg> danboid: that doesn't matter, there's no UI displayed by default
| |
| 19:56 | So having a "Press Esc to abort PXE" or a boot manager, is the same thing, just less pretty
| |
| 19:56 | <mwalters> yah
| |
| 19:56 | I recall it took me forever to get it to load WDS correctly, but failling back to the local bootloader always worked
| |
| 19:56 | <danboid> I've read grub2 can't be chailoader from windows bm
| |
| 19:57 | although it works te other way ofc
| |
| 19:57 | <alkisg> *windows* does not support loading .efi apps
| |
| 19:57 | It just supports special, windows, efi-like apps
| |
| 19:57 | <mwalters> Yeah, I've always done grub then windows
| |
| 19:57 | <alkisg> So it's windows that's broken there
| |
| 19:57 | <mwalters> or pxelinux then windows
| |
| 19:57 | I've never seen windows > anything else, work
| |
| 19:58 | <alkisg> Maybe some time in the future grub or ipxe will release a special, windows-specific efi build,but I don't think that's soonish
| |
| 19:58 | As it will need to rely on windows-specific api
| |
| 19:58 | <danboid> We currently have the Windows boot manaher menu present Ubuntu Linux which loads grub4dos which pxe boots LTSP
| |
| 19:58 | <alkisg> !win32
| |
| 19:58 | <ltsp> I do not know about 'win32', but I do know about these similar topics: 'win32-loader'
| |
| 19:58 | <alkisg> !win32-loader
| |
| 19:58 | <ltsp> win32-loader: win32-loader.exe adds a "PXE - Network boot" option to the Windows boot manager. It's developed for Debian and it uses http://ipxe.org for the network booting. Download it from http://ftp.debian.org/debian/tools/win32-loader/stable. More info at http://goo.gl/FPD7Zv
| |
| 19:58 | <alkisg> This does the same thing, just easier + prettier
| |
| 19:59 | Again, BIOS only, but no need for the ancient grub4dos
| |
| 19:59 | It's using recent grub2
| |
| 20:00 | <danboid> I was going to use refind to boot EFI machines
| |
| 20:00 | I was thinking about using refind
| |
| 20:00 | <alkisg> Refind manages efi entries, but does it have its own boot manager?
| |
| 20:01 | <danboid> I think so yes, a menu
| |
| 20:01 | <alkisg> And why that, instead of grub?
| |
| 20:02 | <danboid> Apparently grub2 and win32-loader don't support UEFI
| |
| 20:02 | <alkisg> Nonsense
| |
| 20:02 | grub2 even supports secure boot
| |
| 20:02 | <danboid> Sorry
| |
| 20:02 | <alkisg> And if you install ubuntu on uefi, it works automatically
| |
| 20:02 | <danboid> We have to use the windows bm
| |
| 20:02 | <alkisg> But refind doesn't use it
| |
| 20:02 | It is its own boot manager, as you say
| |
| 20:02 | kjackal has left IRC (kjackal!~quassel@2a02:587:3101:f300:18fa:1162:c324:7650, Ping timeout: 240 seconds) | |
| 20:03 | <alkisg> So if you're going to use some other boot manager, why not grub, which is widely adopted
| |
| 20:03 | kjackal has joined IRC (kjackal!~quassel@80.107.182.228) | |
| 20:03 | <alkisg> And supports secure boot and everything
| |
| 20:03 | <mwalters> (you just need the signed grub efi file)
| |
| 20:03 | <danboid> I've read a forum post that said it is possible to boot from windows bm to refind to LTSP, I hae yet to try it
| |
| 20:04 | Well, the first two bits - it was to PXE
| |
| 20:04 | <mwalters> I still think it'd be easier to do pxe/grub then the windows bm
| |
| 20:04 | <danboid> Easier yes, but yhos is a primarilt Wondows shop
| |
| 20:04 | primarily Windows
| |
| 20:04 | <mwalters> windows is installed on the clients?
| |
| 20:04 | <danboid> All of them
| |
| 20:05 | <mwalters> offer pxe or grub via dhcp, fallback to the local bootmanager if it times out
| |
| 20:05 | this is what I did way back when
| |
| 20:06 | I even had this working on a win2003 dhcp server
| |
| 20:07 | had a nice menu to pick dban, WDS, etc.
| |
| 20:07 | <danboid> So what does that boot process look like?
| |
| 20:08 | spaced0ut has left IRC (spaced0ut!~spaced0ut@unaffiliated/spaced0ut, Quit: Leaving) | |
| 20:08 | <mwalters> iirc: I had the clients to attempt pxe first, they'd get pxelinux from dhcp, present a menu, the menu had a counter and if it timed out, it just boot from the local disk
| |
| 20:08 | this was back in like 2010, we were just moving from winxp machines to win7 machines at this place
| |
| 20:09 | alkisg: could better talk about how to hook in ltsp from a pxelinux menu, I'm sure
| |
| 20:09 | I don't see why the same couldnt' be done from a grubmenu w/ efi clients
| |
| 20:19 | <danboid> alkisg, So there currently no known way to boot to PXE from the Windows BM on UEFI machines?
| |
| 20:20 | We have to use grub as the boot loader or a PXE menu to get this working on UEFI machines?
| |
| 20:31 | <mwalters> I mean, it's more likely no one has ever really tried that
| |
| 20:31 | <danboid> I've just emailed thr developer of refind to hear what he says
| |
| 20:35 | <mwalters> oh, you know... there is an ipxe efi build, I thought
| |
| 20:35 | you could probably load that with refind
| |
| 20:36 | I had it on a usb when I was goofing with a couple uefi clients that wouldn't cooperate
| |
| 20:36 | <alkisg> danboid: afaik, only ms solutions support pxe from the windows bm
| |
| 20:36 | Like, for WDS etc
| |
| 20:36 | I don't know of any open source solution that ships a windows-api-based .efi file for netbooting
| |
| 20:37 | The problem is in windows, not in the other solutions; windows need solutions to use windows-api
| |
| 20:37 | Maybe some open source app already does that, and I don't know about it
| |
| 20:38 | You can surely search/google, and if you end up finding something, leave us a clue
| |
| 20:38 | I'm more interested in solutions that aren't stricktly windows-based, as uefi doesn't need rely on the windows boot manager anyway
| |
| 20:38 | It's not like bios, which didn't include a boot manager. UEFI does have its own boot manager.
| |
| 20:39 | So you can actually just drop a kernel or ipxe.efi there, and it will be bootable
| |
| 20:40 | <mwalters> it's been a while since I've had a dualboot system, but grub2 has no problems handing off to the windows bm, iirc
| |
| 20:40 | <danboid> Yes but we can't have users using the EFI menus. The springboard in out case has to be wbm
| |
| 20:41 | <alkisg> mwalters: sure, grub2 can chain to windows just fine, it's the opposite that's the issue
| |
| 20:41 | <mwalters> yeah, I get that. I just don't fully understand why he can't do it the "tried and true" way
| |
| 20:41 | just set the grub menu to a 1 second timeout or something
| |
| 20:41 | <alkisg> I don't know why danboid insists on that, yeah
| |
| 20:41 | If tools don't exist, insisting doesn't help :)
| |
| 20:42 | <danboid> I dont think the MS deployment tools support deploying grub y'see
| |
| 20:42 | <alkisg> danboid: no, booting from pxe
| |
| 20:42 | If you boot from pxe, then you can chainload snponly.efi, and that then can load a kernel
| |
| 20:42 | snponly.efi is ipxe loaded from tftp
| |
| 20:43 | <danboid> and where does the Windows bm come into that?
| |
| 20:43 | <alkisg> I mean: windows bm => some windows pxe tool => boot from pxe => snponly.efi => kernel
| |
| 20:43 | <danboid> Ah right!
| |
| 20:43 | Thanks!
| |
| 20:43 | <alkisg> I don't know if such a tool exists
| |
| 20:43 | It existed as a floppy 15 years ago :D
| |
| 20:43 | <danboid> I'm sure the refind dev will know
| |
| 20:44 | <mwalters> curious how this pans out, let us know
| |
| 20:44 | <danboid> Will do
| |
| 20:44 | <alkisg> In http://www.rodsbooks.com/refind/ I see that refind loads windows
| |
| 20:44 | <mwalters> I just recall the only way I could get all the things working way back was to use pxelinux, then fallback on the windows bm
| |
| 20:44 | <alkisg> This would imply it's a boot manager on its own, like grub
| |
| 20:44 | <mwalters> it is, yeah
| |
| 20:45 | <alkisg> Not an .efi application to be installed inside the windows BM
| |
| 20:45 | So I don't know why anyone would prefer it over grub
| |
| 20:45 | <mwalters> that menu looks fancy, though!
| |
| 20:45 | mgariepy has left IRC (mgariepy!~mgariepy@ubuntu/member/mgariepy, Quit: Leaving) | |
| 20:45 | <alkisg> Eh, grub supports themes if someone wants them
| |
| 20:45 | <danboid> grub cannot be loader from wbm on UEFI but refind can, so I've read
| |
| 20:46 | <alkisg> I avoid them as they cause issues with certain graphics drivers
| |
| 20:46 | danboid: are you sure you don't mean the opposite there?
| |
| 20:46 | <mwalters> oh, I didn't realize that
| |
| 20:46 | <alkisg> uefi > refind > windows bm?
| |
| 20:46 | <danboid> Lets see what the dev says eh? :D
| |
| 20:46 | <alkisg> I dont think it can go: UEFI > wbm > refind
| |
| 20:47 | <mwalters> now I'm looking at grub themes
| |
| 20:47 | what a mistake
| |
| 20:47 | <danboid> Maybe not. I'm sure if anyone knows the best route for me it'll be him
| |
| 20:47 | <alkisg> It doesn't hurt to ask, but if it's a boot manager, I doubt it
| |
| 20:48 | <mwalters> looks like #refind also exists here
| |
| 20:49 | * vagrantc only ever used refind to network boot certain combinations of apple hardware and macOS | |
| 20:51 | <alkisg> http://www.rodsbooks.com/refind/installing.html#windows
| |
| 20:51 | "At this point, when you reboot, rEFInd should appear as your new default boot program"
| |
| 20:51 | It doesn't offer a menu,it replaces the windows boot manager, if I read this correctly
| |
| 20:52 | I mean, of course after that,it offers a menu, but that's just like grub
| |
| 20:52 | No UEFI > WBM
| |
| 20:53 | http://forum.ipxe.org/showthread.php?tid=10729 => "Windows bootmgr does not support booting any efi applications (.efi binarys) other then the ones provided by Microsoft, at least not that have been found so far."
| |
| 20:57 | And this lists the MS programs for netbooting: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc732351(v=ws.10)
| |
| 20:57 | They don't seem to have one for .efi
| |
| 20:57 | Maybe some third party developer created one...
| |
| 20:58 | <danboid> I'm guessing that the windows config part is like this, no .efi files are involved
| |
| 20:58 | https://docs.microsoft.com/en-us/windows/deployment/configure-a-pxe-server-to-load-windows-pe
| |
| 20:58 | <mwalters> I mean, without looking at that doc, you're not using the windows bm at all
| |
| 20:59 | <danboid> I think bcdedit is used to set wbm config isn't it?
| |
| 20:59 | <mwalters> no, uefi/bios boots over the network
| |
| 21:00 | oh
| |
| 21:00 | this is just loading the windows bm over the network
| |
| 21:00 | and booting winpe
| |
| 21:00 | <danboid> Yeah but prertend its loading ipxe or whatever instead - there no efi files involved there
| |
| 21:01 | <alkisg> It loads the .wmi file there
| |
| 21:01 | A whole ramdisk
| |
| 21:01 | This one is more close to what you want: https://superuser.com/questions/1045279/use-bcdedit-to-configure-pxe-boot-as-default-boot-option
| |
| 21:01 | It hints that there might be a way to invoke the firmware efi boot option, from the windows boot manager
| |
| 21:02 | Like for example, grub has a "fwsetup" command that chains to the firmware settings,like pressing f2 when it boots
| |
| 21:02 | So if you enable pxe at UEFI, so that this option is available,it might be possible to chain to it
| |
| 21:02 | I.e. pxe enabled, but not default
| |
| 21:02 | <danboid> Yes - that sounds like what we might want here
| |
| 21:03 | <alkisg> The boot path might be different for each board,so you'd need to find the command on a per-client basis
| |
| 21:03 | <danboid> Ah yes - bummer
| |
| 21:03 | Hopefully they don't vary too much
| |
| 21:03 | between manufactuees at least
| |
| 21:06 | <alkisg> On my pc: efibootmgr -v
| |
| 21:06 | Boot0008* UEFI: IP4 Realtek PCIe GBE Family Controller PciRoot(0x0)/Pci(0x1c,0x2)/Pci(0x0,0x0)/MAC(74d435e9b424,0)/IPv4(0.0.0.00.0.0.0,0,0)AMBO
| |
| 21:06 | Maybe you could put that to bcdedit, and chain to it
| |
| 21:06 | <danboid> Hopefully yes
| |
| 21:06 | <alkisg> Maybe you could find the path by booting into a linux live cd and running efibootmgr -v
| |
| 21:06 | <danboid> Good idea! Thanks alkisg!
| |
| 21:07 | <alkisg> np, do tell if it works out
| |
| 21:07 | <danboid> I'll document it for us yes
| |
| 21:18 | <alkisg> vagrantc: about ipv6 local addresses: if we pass ltsp.server=fe80::a9ae:ebb8:f1bb in a client cmdline, then that client can contact the server, wget lts.conf, connect nbd, with a plain `ifup` without even requiring dhcp or an ipv4 address
| |
| 21:19 | Isn't it a good idea to default to that, then?
| |
| 21:19 | For wget=lts.conf and for nbd
| |
| 21:19 | Later on, the OS can do DHCP and get an ipv4 address, as long as it doesn't ifdown the interface
| |
| 21:20 | (the ipv6 local address is usually derived from the mac address, so pretty static)
| |
| 21:21 | <vagrantc> i vaguely recall issues with ipv6 local addresses not crossing certain boundaries ... chained switches or something
| |
| 21:22 | spaced0ut has joined IRC (spaced0ut!~spaced0ut@unaffiliated/spaced0ut) | |
| 21:22 | <vagrantc> it is an interesting idea, though
| |
| 21:22 | <mwalters> I have... a ton of those D:
| |
| 21:22 | <alkisg> Thanks, let me ask about that over at #networking
| |
| 21:22 | mwalters: and ipv6 link-local addresses work?
| |
| 21:22 | <mwalters> I don't ipv6 at all here
| |
| 21:23 | <alkisg> Ah ok
| |
| 21:23 | <vagrantc> also curious how it would work with virtual machines
| |
| 21:23 | <alkisg> My VM fat client wgets from the server just fine
| |
| 21:23 | <vagrantc> cool
| |
| 21:23 | <alkisg> The guys at #networking say "sure, as long as your switches work normally"
| |
| 21:24 | <vagrantc> there are a variety of potential common networking configurations
| |
| 21:24 | <mwalters> I seem to be able to ping the ltsp server ipv6 from my laptop
| |
| 21:24 | I have no idea where the server is getting the address from
| |
| 21:24 | but I'm chained behind like... 3 switches at least
| |
| 21:24 | <vagrantc> will also have to make sure to document the firewalling implications ... some firewalls default to disabling inbound ipv6
| |
| 21:24 | <mwalters> probably 4
| |
| 21:25 | book` has left IRC (book`!~book`@68.ip-149-56-14.net, Ping timeout: 240 seconds) | |
| 21:25 | <mwalters> no idea how to tell if it's a link-local address
| |
| 21:25 | <alkisg> I'll see if nbd properly supports ipv6, and if so, I'll try it in a couple of schools
| |
| 21:25 | <quinox> it starts with fe80:
| |
| 21:25 | <vagrantc> routers will have to be configured to pass ipv6 traffic
| |
| 21:25 | <mwalters> it starts with fe
| |
| 21:25 | eer, fe80
| |
| 21:26 | <alkisg> I don't think we'd want link-local over routers
| |
| 21:26 | <quinox> and it has '<link> next to it in ifconfig
| |
| 21:26 | book` has joined IRC (book`!~book`@68.ip-149-56-14.net) | |
| 21:26 | <vagrantc> i mean, if there are any routers between the client and server
| |
| 21:26 | <alkisg> If someone wants it over routers, he'd use the real ipv6 address
| |
| 21:26 | <mwalters> yeah, I'm able to ping it across multiple switches
| |
| 21:26 | <vagrantc> sure
| |
| 21:27 | i've often wondered about the prospects of an ipv6 native ltsp network, so happy to start exploring it :)
| |
| 21:27 | <mwalters> is there much advantage to an ipv6 lan?
| |
| 21:27 | I tend to not use dns and remember IPs =/
| |
| 21:27 | <quinox> better routing, no weird hidden firewall reroutes needed
| |
| 21:27 | <vagrantc> the ipv6 link-local addresses require very little configuration
| |
| 21:28 | <mwalters> hidden firewall reroutes?
| |
| 21:28 | <alkisg> If you put the kernel/initrd in a client, and then you need to point the initramfs to the ltsp server, it's very hard, if you don't have a configurable router to specify next-server
| |
| 21:28 | <vagrantc> there's (not supposed to be) NAT on ipv6 ... which is a mess
| |
| 21:28 | <quinox> I route traffic from our LAN to our datacenter and the other way around. With IPv6 I can write what I mean instead of using a different IP+port which happens to route to the right IP
| |
| 21:28 | <alkisg> And there are no arping utilities in initramfs, to get from mac to ip
| |
| 21:29 | <mwalters> oic, I have site-to-site VPNs here for that ;)
| |
| 21:29 | <alkisg> So, passing the static link-local server ipv6 address is a good way to tell the clients where the server is
| |
| 21:29 | <quinox> I'm already running so many VPNs to access customer sites... don't need any more :D
| |
| 21:29 | <vagrantc> alkisg: where are you planning on passing this? hard-coded in pxelinux.conf or whatever?
| |
| 21:29 | <mwalters> these are on our routers
| |
| 21:30 | probably a different use case
| |
| 21:30 | 10.0 = here, 10.2 = another office, 10.3 = that other office, and so on
| |
| 21:30 | mfw I realized I never used 10.1
| |
| 21:30 | <alkisg> vagrantc: in the kernel cmdline, however the client gets it, grub, ipxe, whatever
| |
| 21:30 | <vagrantc> you can do similar assignments with ipv6, of course, it's just the numbers are bigger by default
| |
| 21:31 | <mwalters> I'm sure I just need to read on it. I'm just getting old and crotchety and don'tw ant to learn new things ;)
| |
| 21:31 | it's quittin time, though... have a good evening everyone
| |
| 21:31 | * alkisg tries to ssh from a fat client to the server via ipv6 link-local... | |
| 21:32 | <alkisg> Good evening mwalters
| |
| 21:32 | <vagrantc> if ipv6 were adopted widely, you'd probably be able to forget more than you need to learn... but that's not likely to happen any time soon
| |
| 21:33 | <alkisg> People at #networking said that ipv6 link-local addresses are available ...everywhere in the last 15 years or so
| |
| 21:34 | Even if the network isn't ipv6 enabled, that is
| |
| 21:34 | Let's hope that's true :)
| |
| 21:34 | <vagrantc> heh
| |
| 21:35 | <alkisg> ssh alkisg@fe80::a9ae:ebb8:f1bb:2677%enp0s17
| |
| 21:35 | ==> yup, works fine!
| |
| 21:35 | <quinox> \o/
| |
| 21:35 | <alkisg> Lets see about nbd...
| |
| 21:36 | nbd-client fe80::a9ae:ebb8:f1bb:2677%enp0s17 -N /opt/ltsp/nbd3 /dev/nbd2 -swap -persist
| |
| 21:36 | Negotiation: ..size = 10238MB
| |
| 21:36 | Yey!
| |
| 21:38 | Timing buffered disk reads: 1058 MB in 3.01 seconds = 352.01 MB/sec
| |
| 21:38 | As fast as ipv4, ok there too
| |
| 21:38 | <vagrantc> https://en.wikipedia.org/wiki/Comparison_of_IPv6_support_in_operating_systems
| |
| 21:38 | looks promising
| |
| 21:39 | <alkisg> No more ipappend 3 vs 2 nightmares :)
| |
| 21:40 | !tftp
| |
| 21:40 | <ltsp> tftp: Here's a page to help you troubleshoot TFTP problems in Ubuntu: https://help.ubuntu.com/community/UbuntuLTSP/Troubleshooting/TFTP
| |
| 21:40 | <vagrantc> i'm sure we'll find other nightmares :)
| |
| 21:41 | <quinox> the only things on my network that doesn't support it is are overpriced Cisco VoIP phones and a bottom-of-the-professional-market network printer
| |
| 21:42 | <alkisg> tftp seems to fail
| |
| 21:42 | <vagrantc> this is from the initramfs?
| |
| 21:42 | <alkisg> Currently from the booted system
| |
| 21:43 | Transfer timed out
| |
| 21:43 | <vagrantc> you might have to quote the address in various different ways depending on the application
| |
| 21:43 | <alkisg> tftp fe80::a9ae:ebb8:f1bb:2677%enp0s17 -v -m binary -c get /ltsp/i386/lts.conf
| |
| 21:43 | It doesn't like []
| |
| 21:43 | Connected to fe80::a9ae:ebb8:f1bb:2677%enp0s17 (fe80::a9ae:ebb8:f1bb:2677), port 69
| |
| 21:43 | I think it connects, but fails to transfer
| |
| 21:43 | <vagrantc> wasn't lts.conf going to be delivered over http anyways?
| |
| 21:44 | <alkisg> Sure, just as a quick upgrade
| |
| 21:44 | for ltsp5
| |
| 21:44 | <quinox> I'm off to bed - let me know if I can help by testing anything on my network
| |
| 21:44 | <alkisg> Thank you quinox
| |
| 21:46 | Ah, I didn't start dnsmasq with the scope id
| |
| 21:47 | So it may not know how to respond to link-local addresses... /me tests....
| |
| 21:49 | Nah,that's the server, not the client, it doesn't need that
| |
| 21:58 | an 30 23:56:01 alkis nbd_server[19592]: connect from fe80::5054:ff:fe25:e337%enp2s0, assigned file is /tmp/nbd-swap/fe80::5054:ff:fe25:e337%enp2s0
| |
| 21:58 | Haha our scripts work fine, the client even got ipv6 swap
| |
| 22:06 | danboid has left IRC (danboid!~dan@cpc126962-macc4-2-0-cust227.1-3.cable.virginm.net, Remote host closed the connection) | |
| 22:06 | <alkisg> Aaand ipv6 ldm login. Yey!
| |
| 22:07 | * alkisg calls it a day, 'night all | |
| 22:09 | ricotz has left IRC (ricotz!~ricotz@ubuntu/member/ricotz, Remote host closed the connection) | |
| 22:48 | vagrantc has left IRC (vagrantc!~vagrant@unaffiliated/vagrantc, Quit: leaving) | |
| 23:22 | ||cw has left IRC (||cw!~chrisw@unaffiliated/cw/x-1182934, Ping timeout: 245 seconds) | |
| 23:27 | ||cw has joined IRC (||cw!~chrisw@unaffiliated/cw/x-1182934) | |
| 23:51 | kjackal has left IRC (kjackal!~quassel@80.107.182.228, Ping timeout: 245 seconds) | |