xKing Posted March 11 Posted March 11 Interface `tap0` should come UP if this thing is set to 1. net.link.tap.up_on_open: 1 It’s done by the `vm init` which is called if you have `vm_enable=YES` in /etc/rc.conf Or you can do that manually by running `sudo vm init` before you do `sudo vm start` See if you can do a couple reboots and just check `sysctl -a | grep up_on_open` after reboot - you always see 1 ?
tazman Posted March 11 Author Posted March 11 [admin@eisy ~]$ sysctl -a | grep up_on_open net.link.tap.up_on_open: 1 [admin@eisy ~]$ sudo ifconfig -a Password: re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> ether 00:21:b9:02:61:da inet6 fe80::221:b9ff:fe02:61da%re0 prefixlen 64 scopeid 0x1 inet6 2601:500:8801:38a0:221:b9ff:fe02:61da prefixlen 64 autoconf inet 10.0.0.66 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> vm-public: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 9e:73:5c:44:25:33 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 4 priority 128 path cost 2000000 member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 1 priority 128 path cost 20000 groups: bridge vm-switch viid-4c918@ nd6 options=9<PERFORMNUD,IFDISABLED> tap0: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: vmnet/homeassistant/0/public options=80000<LINKSTATE> ether 58:9c:fc:10:90:31 groups: tap vm-port media: Ethernet autoselect status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Opened by PID 6388 @xKingI will do a couple of reboots but this is the state it was in when HA did not come online after the last reboot. It seems odd to have the 1 but the devices not to be 'UP'
tazman Posted March 11 Author Posted March 11 Start 1 everything good and HA started Start 2 '1' and vm-public UP at reboot HA would not come up and vm-public and Tap0 not up after starting HA Start 3 '1' and vm-public UP at reboot HA started noticed pattern from before did 'sudo vm poweroff -f homeassistant' then rebooted Start 4 '1' and vm-public UP at reboot HA would not come up and vm-public and Tap0 not up after starting HA Start 5 '1' and vm-public UP at reboot HA started moved vm_dir="zfs:storage/vms" to before vm_enable="YES" in rc.conf file Start 6 '1' and vm-public UP at reboot then HA started Start 7 '1' and vm-public UP at reboot then HA started Start 8 '1' and vm-public UP HA did not work Start 9 '1' and vm-public UP at reboot HA started Start 10 '1' and vm-public UP at reboot HA started @xKingI feel moving the path of vm before starting vm helped but I can test more tomorrow afternoon if you want more restarts or to try something else.
xKing Posted March 11 Posted March 11 Sorry if I was not specific but please watch the `tap0` status. Just 2-3 reboots is ok net.link.tap.up_on_open - is the system variable that says - bring the tap interface UP as soon as there is an opener (VM starts).
rob7419 Posted March 11 Posted March 11 (edited) @xKing and @tazman xKing, I would like to also express my appreciation on your "Guru" assistance. And here is what I have so far: 20 hours ago, xKing said: For the reference - I'm using pfSense 23.09.1-RELEASE as a router with KEA as DHCP server selected. I also am on pfSense 23.09.1 RELEASE, but still using ISC DHCP server, haven't switched to KEA due to it not being fully implemented. Was going to switch when it became the default. I might attempt to switch to KEA and see if it make a difference but will have to wait till later this afternoon to try. -------------------------------------------------------------------------------------------------------- After reading through all the post, the only thing that I changed to start with is in the /etc/rc.conf file: Quote ifconfig_re0_ipv6="inet6 accept_rtadv" to ifconfig_re0_ipv6="accept_rtadv" Didn't make a difference in how the Home Assistant VM started, VM was running but not connect to the network. -------------------------------------------------------------------------------------------------------- I then commented out in /etc/rc.conf Quote #vm_list="homeassistant" Performed 4 consecutive reboots with the following steps. $ ifconfig -a - to check if vm-public was "UP" and in all 4 reboots it was true. $ sysctl -a | grep net.link.tap.up_on_open - check if net.link.tap.up_on_open: was = "1", all 4 reboots it was true. $ sudo vm start homeassistant - in all 4 cases the VM started successfully, the tap0 interface was created with a status of "UP" and connected to the network. Then I uncomment the "vm_list" line above to set /etc/rc.conf back the the original for the VM to start during boot. Again performed 4 consecutive reboots, VM was running but none of the reboots was it able to connect to the network on the initial boot sequence. Check the following on each: $ ifconfig -a - to check if vm-public and tap0 was "UP" and in all 4 reboots it was false. Neither interface showed "UP" status, on reboots 2 & 4 waited about 3 minute and then Quote sudo ifconfig vm-public up; sudo ifconfig tap0 up and the VM immediately started replying to ping request. $ sysctl -a | grep net.link.tap.up_on_open - check if net.link.tap.up_on_open: was = "1", all 4 reboots it was true. -------------------------------------------------------------------------------------------------------- 3 hours ago, xKing said: Sorry if I was not specific but please watch the `tap0` status. Just 2-3 reboots is ok net.link.tap.up_on_open - is the system variable that says - bring the tap interface UP as soon as there is an opener (VM starts). In all the reboots, this variable was always set to "1". When does this variable get set during the boot sequence? Is it possible that the VM is starting before it does and prevents the network from being established? -------------------------------------------------------------------------------------------------------- Still working on tcpdump packet captures. Will DM them when I get something worth sending, will try to capture from pfSense first and then from the eisy. And finally my /etc/rc.conf and VM configure files for you to review: rc_conf 03-11-2024.txt HA_VM_Configure 03-11-2024.txt Edited March 11 by rob7419
xKing Posted March 11 Posted March 11 @rob7419 thank you sir, can you try adding net.link.tap.up_on_open=1 at the bottom of /boot/loader.conf This should set it as early as possible during the boot sequence.
rob7419 Posted March 11 Posted March 11 1 hour ago, xKing said: @rob7419 thank you sir, can you try adding net.link.tap.up_on_open=1 at the bottom of /boot/loader.conf This should set it as early as possible during the boot sequence. Added to /boot/loader.conf - net.link.tap.up_on_open=1 Quote [admin@eisy ~]$ cat /boot/loader.conf kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" cryptodev_load="YES" zfs_load="YES" tpm_load="YES" coretemp_load=YES autoboot_delay="0" kern.cam.boot_delay="1000" beastie_disable="YES" net.link.tap.up_on_open=1 [admin@eisy ~]$ no change and just in case Quote net.link.tap.up_on_open="1" still no change. In both cases Quote sudo ifconfig vm-public up; sudo ifconfig tap0 up Brought the Home Assistant VM network back online.
tazman Posted March 11 Author Posted March 11 @xKingI think it is safe to say net.link.tap.up_on_open: 1 and vm-public is UP at start are being set correctly but unlike @rob7419 I have experienced starting HA manually still results in a fail and vm-public and Tap0 are not showing as UP after starting HA but sudo ifconfig vm-public up; sudo ifconfig tap0 up sets them to UP and HA starts working.
xKing Posted March 12 Posted March 12 Allright, remove that line from loader.conf and add these (just for the TEST) to /etc/rc.conf Reboot and see if homeassistant still has an issue? udx_enable="NO" isy_enable="NO"
tazman Posted March 12 Author Posted March 12 @xKingrebooted 3 times and HA came right up ###MAINTAINED BY UDX#### ###DO NOT MODIFY### hostname="eisy" # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable dumpdev="NO" zfs_enable="YES" dbus_enable="YES" sshd_enable="YES" moused_nondefault_enable="NO" sendmail_submit_enable="NO" sendmail_outbound_enable="NO" sendmail_msp_queue_enable="NO" clear_tmp_enable="YES" udx_enable="NO" isy_enable="NO" #NTP ntpd_enable="YES" ntpd_sync_on_start="YES" #HDMI is disabled by default #kld_list="i915kms" #Make sure dhclient is run in the background background_dhclient="YES" #make sure we support double stack ipv6_ipv4mapping="YES" defaultroute_delay="0" #Network Interfaces (default) rtsold_enable="YES" #for now, do not let iwlwifi/iwm get loaded devmatch_enable="YES" devmatch_blocklist="if_iwm if_iwlwifi" UDX_UPGRADE_STATUS="active" ifconfig_re0_ipv6="accept_rtadv" ifconfig_re0="DHCP" vm_dir="zfs:storage/vms" #ifconfig_re0_ipv6="inet6 accept_rtadv" vm_enable="YES" #vm_dir="zfs:storage/vms" vm_list="homeassistant" mdnsresponderposix_flags="-f /usr/local/etc/udx.d/static/mdns.services" ~
rob7419 Posted March 12 Posted March 12 29 minutes ago, xKing said: Allright, remove that line from loader.conf and add these (just for the TEST) to /etc/rc.conf Reboot and see if homeassistant still has an issue? udx_enable="NO" isy_enable="NO" Me too, 3 successfully reboots with HA VM connecting to the network.
xKing Posted March 12 Posted March 12 Thanks! Time to ask @Michel Kohanim what kind of shady business is he up to in those scripts. There is a fun thing to do - reboot with UDX and ISY disabled and do dmesg look at last lines where interfaces go UP/DOWN now re-enable UDX and ISY - reboot again and just do dmesg command again - see more UP/DOWN stuff?
tazman Posted March 12 Author Posted March 12 31 minutes ago, xKing said: Thanks! Time to ask @Michel Kohanim what kind of shady business is he up to in those scripts. There is a fun thing to do - reboot with UDX and ISY disabled and do dmesg look at last lines where interfaces go UP/DOWN now re-enable UDX and ISY - reboot again and just do dmesg command again - see more UP/DOWN stuff? This is the end of the file without ISY and UDX WARNING: attempt to domain_add(bluetooth) after domainfinalize() WARNING: attempt to domain_add(netgraph) after domainfinalize() bridge0: Ethernet address: EDITED bridge0: changing name to 'vm-public' re0: promiscuous mode enabled vm-public: link state changed to UP Security policy loaded: MAC/ntpd (mac_ntpd) tap0: Ethernet address: EDITED tap0: promiscuous mode enabled re0: link state changed to DOWN vm-public: link state changed to DOWN tap0: link state changed to UP vm-public: link state changed to UP re0: link state changed to UP This is ISY and UDX enabled then reboot and HA not working WARNING: attempt to domain_add(bluetooth) after domainfinalize() WARNING: attempt to domain_add(netgraph) after domainfinalize() bridge0: Ethernet address: EDITED bridge0: changing name to 'vm-public' re0: promiscuous mode enabled vm-public: link state changed to UP Security policy loaded: MAC/ntpd (mac_ntpd) tap0: Ethernet address: EDITED tap0: promiscuous mode enabled re0: link state changed to DOWN vm-public: link state changed to DOWN tap0: link state changed to UP vm-public: link state changed to UP re0: link state changed to UP lo0: link state changed to DOWN lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP I have both full files saved if they are needed.
tazman Posted March 12 Author Posted March 12 10 minutes ago, Michel Kohanim said: Is anyone using WiFi? With kind regards, Michel I am not using WIFI.
rob7419 Posted March 12 Posted March 12 8 hours ago, xKing said: Thanks! Time to ask @Michel Kohanim what kind of shady business is he up to in those scripts. There is a fun thing to do - reboot with UDX and ISY disabled and do dmesg look at last lines where interfaces go UP/DOWN now re-enable UDX and ISY - reboot again and just do dmesg command again - see more UP/DOWN stuff? Just to verify that I'm getting the simular results as @tazman UDX & ISY Disabled, HA VM connects to network Quote Trying to mount root from zfs:zudi/ROOT/default []... uhub0: 14 ports with 14 removable, self powered Root mount waiting for: usbus0 CAM ugen0.2: <vendor 0x8087 product 0x0026> at usbus0 lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP ubt0 on uhub0 ubt0: <vendor 0x8087 product 0x0026, class 224/1, rev 2.01/0.02, addr 1> on usbus0 bridge0: Ethernet address: 58:9c:fc:**:**:** bridge0: changing name to 'vm-public' re0: promiscuous mode enabled vm-public: link state changed to UP Security policy loaded: MAC/ntpd (mac_ntpd) tap0: Ethernet address: 58:9c:fc:**:**:** tap0: promiscuous mode enabled re0: link state changed to DOWN vm-public: link state changed to DOWN tap0: link state changed to UP vm-public: link state changed to UP re0: link state changed to UP UDX & ISY Enabled, HA VM will not connect to network, last couple of lines on mine are different Quote Trying to mount root from zfs:zudi/ROOT/default []... uhub0: 14 ports with 14 removable, self powered Root mount waiting for: usbus0 ugen0.2: <vendor 0x8087 product 0x0026> at usbus0 lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP ubt0 on uhub0 ubt0: <vendor 0x8087 product 0x0026, class 224/1, rev 2.01/0.02, addr 1> on usbus0 bridge0: Ethernet address: 58:9c:fc:**:**:** bridge0: changing name to 'vm-public' re0: promiscuous mode enabled vm-public: link state changed to UP Security policy loaded: MAC/ntpd (mac_ntpd) tap0: Ethernet address: 58:9c:fc:**:**:** re0: link state changed to DOWN vm-public: link state changed to DOWN tap0: promiscuous mode enabled tap0: link state changed to UP vm-public: link state changed to UP re0: link state changed to UP lo0: link state changed to DOWN lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP tap0: link state changed to DOWN Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 0 0 0 0 done All buffers synced. 7 hours ago, Michel Kohanim said: Is anyone using WiFi? With kind regards, Michel I am on a wired connection.
Michel Kohanim Posted March 12 Posted March 12 It's very odd. We do use tap0 for Wifi. ISY has no permissions to do anything with the hardware/networking. If you have ever attempted to use wifi, please do this: sudo udxops.sh wifi.reset Also, please do make sure in /etc/rc.conf: ifconfig_inet6="inet6 accept_rtadv" (you need the inet6 in there. With kind regards, Michel
rob7419 Posted March 12 Posted March 12 This is the dmesg -a output with both UDX & ISY enabled and #vm_list="homeassistant" disabled in /etc/rc.conf, once eisy boots, start the HA VM. Quote Trying to mount root from zfs:zudi/ROOT/default []... uhub0: 14 ports with 14 removable, self powered Root mount waiting for: usbus0 ugen0.2: <vendor 0x8087 product 0x0026> at usbus0 lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP ubt0 on uhub0 ubt0: <vendor 0x8087 product 0x0026, class 224/1, rev 2.01/0.02, addr 1> on usbus0 bridge0: Ethernet address: 58:9c:fc:**:**:** bridge0: changing name to 'vm-public' re0: promiscuous mode enabled vm-public: link state changed to UP Security policy loaded: MAC/ntpd (mac_ntpd) *** sudo vm start homeassistant *** addition to dmesg -a once HA VM is started and successfully connects to network tap0: Ethernet address: 58:9c:fc:**:**:** tap0: promiscuous mode enabled re0: link state changed to DOWN vm-public: link state changed to DOWN tap0: link state changed to UP vm-public: link state changed to UP re0: link state changed to UP The interfaces do not produce the same behavior when started separately. 33 minutes ago, Michel Kohanim said: It's very odd. We do use tap0 for Wifi. ISY has no permissions to do anything with the hardware/networking. If you have ever attempted to use wifi, please do this: sudo udxops.sh wifi.reset Also, please do make sure in /etc/rc.conf: ifconfig_inet6="inet6 accept_rtadv" (you need the inet6 in there. With kind regards, Michel Reset the Wifi, I've never set up the WIFI, but just to make sure this wasn't the issue. HA VM went offline when $ sudo udxops.sh wifi.reset was entered. Updated back to: ifconfig_inet6="inet6 accept_rtadv" in /etc/rc.conf and enabled the vm_list="homeassistant" to enabled VM to start on boot. Rebooted eisy, dmesg -a output with same results that HA VM starts but does not connect to the network. Quote Trying to mount root from zfs:zudi/ROOT/default []... uhub0: 14 ports with 14 removable, self powered Root mount waiting for: usbus0 CAM ugen0.2: <vendor 0x8087 product 0x0026> at usbus0 lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP ubt0 on uhub0 ubt0: <vendor 0x8087 product 0x0026, class 224/1, rev 2.01/0.02, addr 1> on usbus0 bridge0: Ethernet address: 58:9c:fc:**:**:** bridge0: changing name to 'vm-public' re0: promiscuous mode enabled vm-public: link state changed to UP Security policy loaded: MAC/ntpd (mac_ntpd) tap0: Ethernet address: 58:9c:fc:**:**:** tap0: promiscuous mode enabled re0: link state changed to DOWN vm-public: link state changed to DOWN tap0: link state changed to UP vm-public: link state changed to UP re0: link state changed to UP lo0: link state changed to DOWN lo0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP No change in boot with WiFi reset.
tazman Posted March 12 Author Posted March 12 4 minutes ago, rob7419 said: No change in boot with WiFi reset. To bad I had high hopes my eisy started out as WIFI but I did the 4 button press to reset when we found the problem with some node servers and WIFI. I will still give it a try tonight when I get home.
Michel Kohanim Posted March 12 Posted March 12 Please send me /var/udx/logs/debug.log With kind regards, Michel
rob7419 Posted March 12 Posted March 12 40 minutes ago, Michel Kohanim said: Please send me /var/udx/logs/debug.log With kind regards, Michel Sent them to support@universal-devices.com
rob7419 Posted March 12 Posted March 12 @tazman Just to keep you updated also, Michel responded to my email "Looking through the code … I’ll keep you posted."
tazman Posted March 13 Author Posted March 13 @rob7419thanks for the update I sent Michel a message through my ticket because I was not able to figure out how to access the file he wanted.
apnar Posted March 13 Posted March 13 On 3/8/2024 at 5:57 PM, rob7419 said: @Michel Kohanim I am not able to get to any other CLI session other then the HA CLI through the VNC viewer. FYI, to get out of "ha" prompt and down to real OS you can run "login". That will drop you to a proper linux prompt in the HA OS.
Recommended Posts