macjeff Posted July 4, 2022 Share Posted July 4, 2022 This AM I ran updates via ssh and there was over 100 freebsd updates. About 1-2 hours ago I tried to SSH in and got this message macjeff@MacBook-Pro-Retina-2018 ~ % ssh admin@172.16.1.210kex_exchange_identification: Connection closed by remote hostConnection closed by 172.16.1.210 port 22 I tried many times. I checked my IP which is reserved DHCP above the normal DHCP range given out. Everything else was working so I did backup of ISY and PG3. Confirmed nothing else is using that IP. So I rebooted polisy. I can ping it but now but cant get in Pg3, ISY admin, or SSH Well SSH now goes in but wont take my password or the default admin password so I dont think I am really in So essentially locked out. About to try the reset options. Link to comment
Bumbershoot Posted July 4, 2022 Share Posted July 4, 2022 This sounds like all the updates filled all your disk space. My Polisy shipped with only a 4gb partition, and a few large updates filled it up, leaving the Polisy to present similar symptoms. After a few hours of letting it sit, I was able log in, and I ran the command ‘sudo pkg clean’, which solved the problem. Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 Well I bought a new polisy as a backup a few months ago. Hooked that up. Firmware was on 5.2.2 so I ran Pkg update and pkg upgrade. Again it downloaded a lot of updates and again it finished and then when I rebooted I am locked out Something is going on here. Now I am totally down with two units. Link to comment
Bumbershoot Posted July 4, 2022 Share Posted July 4, 2022 4 minutes ago, macjeff said: Well I bought a new polisy as a backup a few months ago. Hooked that up. Firmware was on 5.2.2 so I ran Pkg update and pkg upgrade. Again it downloaded a lot of updates and again it finished and then when I rebooted I am locked out Something is going on here. Now I am totally down with two units. Open a support ticket. Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 I did. Already got a response they had not seen this. But now I have seen it twice. And one was a brand new polisy out of the box. I was forced to run updates to get the firmware and everything up to date. Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 on both polisy now. Both reset so the password is just admin Last login: Mon Jul 4 01:04:11 on ttys000 macjeff@MacBook-Pro-Retina-2018 ~ % ssh admin@172.16.1.210 ssh: connect to host 172.16.1.210 port 22: Connection refused macjeff@MacBook-Pro-Retina-2018 ~ % or this macjeff@MacBook-Pro-Retina-2018 ~ % ssh admin@172.16.1.61 (admin@172.16.1.61) Password for admin@polisy: (admin@172.16.1.61) Password for admin@polisy: (admin@172.16.1.61) Password for admin@polisy: admin@172.16.1.61: Permission denied (publickey,keyboard-interactive). macjeff@MacBook-Pro-Retina-2018 ~ % Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 (edited) there has to be another way into the Polisy. I tried a USB monitor but no drivers it does power up but no drivers. I have a ticket in and talking to UD tomorrow. Whatever updates are coming out from freeBSD is what did both units in. And one was right out of the box. If I had another chance I would update one by one only what is needed but the problem is out of the box the freebsd is old version 13 and all the other packages from UDI are all old. You would think a reset would load a flashed OS to restart as new but it only clears settings. Oh both units Ping just fine so they are online. Just not responding to SSH or any other method I have tried. Edited July 4, 2022 by macjeff Link to comment
mwester Posted July 4, 2022 Share Posted July 4, 2022 There's an internal serial console port. I wrote a "How To" post describing how to find the connector and gain access to this, but it's not showing up in the searches, so it's either been removed or locked down. Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 1 hour ago, mwester said: There's an internal serial console port. I wrote a "How To" post describing how to find the connector and gain access to this, but it's not showing up in the searches, so it's either been removed or locked down. what cable would I need? Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 1 hour ago, mwester said: There's an internal serial console port. I wrote a "How To" post describing how to find the connector and gain access to this, but it's not showing up in the searches, so it's either been removed or locked down. On another forum other users are reporting this issue. I asked them to post here. I think its a freebsd issue that prevents it from booting. One of my units it is not even listening on port 22 for SSH. the other responds but wont accept the password even after reset. Link to comment
Bumbershoot Posted July 4, 2022 Share Posted July 4, 2022 2 hours ago, macjeff said: I think its a freebsd issue that prevents it from booting. 4 hours ago, macjeff said: Whatever updates are coming out from freeBSD is what did both units in. Did you have the BIOS updated to EUFI on your primary Polisy prior to the last software update (seems sensible that EUFI wouldn't have been installed on your backup machine quite yet)? I wonder if there's a issue with the newest packages from UDI and a BIOS that hasn't been upgraded... I've got my Polisy running the latest/greatest FreeBSD release from UDI, with all the packages updated, BIOS updated to EUFI and TPM running, and it's running fine. I'd think there would be hundreds of boxes down if somehow the FreeBSD updates were broken is some way. 2 hours ago, macjeff said: On another forum other users are reporting this issue. I asked them to post here. I hope someone does report the issue. Very curious about this one. Would you please update the thread with the resolution? Thanks. Link to comment
macjeff Posted July 4, 2022 Author Share Posted July 4, 2022 8 minutes ago, Bumbershoot said: Did you have the BIOS updated to EUFI on your primary Polisy prior to the last software update (seems sensible that EUFI wouldn't have been installed on your backup machine quite yet)? I wonder if there's a issue with the newest packages from UDI and a BIOS that hasn't been upgraded... I've got my Polisy running the latest/greatest FreeBSD release from UDI, with all the packages updated, BIOS updated to EUFI and TPM running, and it's running fine. I'd think there would be hundreds of boxes down if somehow the FreeBSD updates were broken is some way. I hope someone does report the issue. Very curious about this one. Would you please update the thread with the resolution? Thanks. I agree with you. There has been two other reports. It may have been the timing of when I did this and they already fixed the issue online. The fix is going to be something UDI has to help with. Link to comment
mwester Posted July 4, 2022 Share Posted July 4, 2022 6 hours ago, macjeff said: what cable would I need? I used a USB-to-Serial converter with TTL-level serial pinouts -- the one sold by Adafruit, if I recall correctly. I do not have the photos and documentation on my system any longer (I no longer have a Polisy, so I probably left those behind when I upgraded my laptop last fall). The serial port is a set of 4 or 5 pins near the edge of the board, on a white connector as I recall. You'll need only three of them (txd, rxd, and gnd). The documentation for the motherboard is available online from the board manufacturer, which is where I got it. Link to comment
Michel Kohanim Posted July 5, 2022 Share Posted July 5, 2022 Hello all, The issue is that we migrated all ssh pkg commands to the Admin Console to ensure the correct sequence of things are done. For instance, migrating from 12 to 13 required all those scripts (10% failed). Upgrade from 13 to 13.1 has a bug that corrupts the password file. By using the button in the Admin Console, the system will figure out which scripts should be run to avoid all these problems. The biggest issue was that, except a mention in release notes, I didn't advertise this fact. Given the millions of times we asked everyone to use pkg update/upgrade, OS version 13 was moved to 13.1 and thus corrupted the password file. The fix is rather simple using what was described by @mwester. Sincere apologies for the inconvenience. With kind regards, Michel 1 Link to comment
Bumbershoot Posted July 5, 2022 Share Posted July 5, 2022 17 minutes ago, Michel Kohanim said: By using the button in the Admin Console, the system will figure out which scripts should be run to avoid all these problems. That's easy enough to understand. That'll be my new practice. 1 Link to comment
JBanaszak Posted July 5, 2022 Share Posted July 5, 2022 36 minutes ago, Michel Kohanim said: Hello all, The issue is that we migrated all ssh pkg commands to the Admin Console to ensure the correct sequence of things are done. For instance, migrating from 12 to 13 required all those scripts (10% failed). Upgrade from 13 to 13.1 has a bug that corrupts the password file. By using the button in the Admin Console, the system will figure out which scripts should be run to avoid all these problems. The biggest issue was that, except a mention in release notes, I didn't advertise this fact. Given the millions of times we asked everyone to use pkg update/upgrade, OS version 13 was moved to 13.1 and thus corrupted the password file. The fix is rather simple using what was described by @mwester. Sincere apologies for the inconvenience. With kind regards, Michel Thanks Michel. I am actually moving next week. I will leave the ISY behind to run a few automated lighting programs but take the Polisy with me to run IoP and PG3 at another home. So far the Polisy/PG2 seems to be running fine but I can no longer SSH in. I can login to IoP normally. Do I need to somehow reset the Polisy after moving or will this sort itself out on the next update? Link to comment
Geddy Posted July 5, 2022 Share Posted July 5, 2022 @mwester - Here's the post you were mentioning. Thanks to @MrBill for remembering where it might have been and letting me know. It was in the old "Geek Batch" area that had been removed. Since this has come up again and might be needed for a reference point it's available for review. Just no more posts are possible in that section. Thanks @MrBill for the help and @mwester for this great post to refer back to! 1 Link to comment
Michel Kohanim Posted July 6, 2022 Share Posted July 6, 2022 21 hours ago, JBanaszak said: I can login to IoP normally. Do I need to somehow reset the Polisy after moving or will this sort itself out on the next update? @JBanaszak, this is very very strange. Have you tried clicking on the Upgrade Packages button in the Admin Console? With kind regards, Michel Link to comment
MrBill Posted July 6, 2022 Share Posted July 6, 2022 21 hours ago, JBanaszak said: but I can no longer SSH in. do you mean that SSH doesn't connect, or that your login credentials are not accepted? Link to comment
macjeff Posted July 6, 2022 Author Share Posted July 6, 2022 SSH wont work if you SSH'd to do the updates. I have been doing SSH here for so many years. Read the post from Michel. He will probably have to help you get back in as he is me. Things will continue to work unless you reboot the system so dont make my mistake and talk to him before rebooting. As he said, and I have learned my lesson, from now on use the admin console to do all updates. No more SSH for updates. Link to comment
hart2hart Posted July 6, 2022 Share Posted July 6, 2022 On 7/5/2022 at 10:11 AM, Michel Kohanim said: By using the button in the Admin Console, the system will figure out which scripts should be run to avoid all these problems. To confirm, is this the button from within IoP which I'm not using yet and I've not been into the specific Polisy config screen since I set the box up? Link to comment
JBanaszak Posted July 7, 2022 Share Posted July 7, 2022 On 7/6/2022 at 8:40 AM, Michel Kohanim said: @JBanaszak, this is very very strange. Have you tried clicking on the Upgrade Packages button in the Admin Console? With kind regards, Michel Michel, I did click the button. Admin Console closed immediately but no other obvious event happened. Link to comment
JBanaszak Posted July 7, 2022 Share Posted July 7, 2022 On 7/6/2022 at 8:43 AM, MrBill said: do you mean that SSH doesn't connect, or that your login credentials are not accepted? The former: SSH does not connect. I can login to PG2 through a web browser as usual and connect to IoP using my credentials in the Admin Console. Link to comment
Bumbershoot Posted July 7, 2022 Share Posted July 7, 2022 27 minutes ago, JBanaszak said: The former: SSH does not connect. I can login to PG2 through a web browser as usual and connect to IoP using my credentials in the Admin Console. Michel indicated that there could be corruption of the /etc/passwd file if the updates weren't done in a specific order (which is what using the "Upgrade Packages" button on the AC sorts out for us). I don't think this is a problem specific to FreeBSD at all (there are no new entries in the FreeBSD bugzilla site for 'passwd'), but something specific to what UDI is doing on their implementation. Link to comment
macjeff Posted July 7, 2022 Author Share Posted July 7, 2022 24 minutes ago, Bumbershoot said: Michel indicated that there could be corruption of the /etc/passwd file if the updates weren't done in a specific order (which is what using the "Upgrade Packages" button on the AC sorts out for us). I don't think this is a problem specific to FreeBSD at all (there are no new entries in the FreeBSD bugzilla site for 'passwd'), but something specific to what UDI is doing on their implementation. He told me its the new Python doing it Link to comment
Recommended Posts