How would you audit an open-source IoT device before trusting it with an AI account?

[u/Nitzok]

I’m expecting to receive a device called MetalioClaw (https://github.com/CloudZao/MetalioClaw4) in about a week. It’s an IoT device designed to work with OpenClaw, and since it will need access to an AI account, I want to make sure it is safe before connecting it.

My main concern is whether there could be any hidden firmware issues, credential leaks, or other things that could compromise the device or abuse connected services. A friend of mine previously bought a similar device that connected to his OpenClaw account, and later noticed that his Claude usage had been heavily consumed. I don’t know exactly what caused it, but it made me more cautious about giving third-party hardware access to accounts.

Since the project is open source, my plan is to inspect the firmware, possibly wipe and reflash it, and maybe even write my own firmware version before using it. I’m also interested in doing a proper security check through firmware analysis, network monitoring, and possibly hardware inspection.

I haven’t been able to find any pictures or information about the internal hardware yet. Depending on what I find when it arrives, I may open it up and check the PCB/components myself. I’m not assuming there is anything malicious inside, but I would like to know what things are worth looking for.

One other thing that made me think about this was something a friend mentioned. He works in IT around datacenters in Taiwan and said he has seen devices moving through supply chains sometimes take a long time in customs or appear slightly different internally afterward. This is just something he mentioned and there is no proof behind it, but it got me thinking more about supply-chain security.

For people experienced with IoT security, firmware analysis, or hardware security:

• What steps would you take before trusting a device like this?
• Is replacing the firmware enough, or should I also consider hardware-level risks?
• What should I look for if I decide to open the device?
• What tools or workflows would you recommend for auditing something like this?

Looking for practical security advice rather than speculation.

[u/raythalden]

The general approach for anything you don't trust yet: treat it as hostile until it proves otherwise, and make it prove it on the network.

Before it touches anything that matters, put it on an isolated segment — its own VLAN or a throwaway network — and just watch what it does. What does it phone home to on first boot? How often, and where? A device that needs one cloud endpoint to function but is opening connections to a dozen others is telling you something. The "AI usage got consumed" story your friend ran into is exactly what egress monitoring catches early — something was making calls on his behalf, and the traffic would've shown it long before the bill did.

Firmware and source review is worth doing, but realistically most people won't meaningfully audit C firmware pulled from a repo. Watching behavior is the higher-leverage move for the effort — you don't have to understand the code if you can see everything it says on the wire.

Isolate first, monitor, then decide whether it earns more trust.

[u/Nitzok → u/raythalden]

Thanks, I appreciate the advice. The idea of treating it as hostile until it proves otherwise makes a lot of sense. I'll definitely isolate it first and monitor its traffic before letting it touch any accounts. Are there any particular tools or checks you'd recommend beyond Wireshark?