Troubleshoot Plc Exception

Why a PFC200 stopped

The runtime writes down exactly why it stopped, and the record survives a restart. Almost all the time lost to a failure like this goes on not reading it: the file is not where the CODESYS documentation implies, the controller's clock is years wrong so its timestamps look irrelevant, and the LEDs on the front invite a diagnosis they cannot support.

This page is the route to the evidence, in the order that gets there fastest.

1. Establish what was actually dead

Do this before touching the controller, because it decides which half of the system to investigate and the answers are all free:

-R is not optional. It suppresses retained messages, and the retained value of an availability topic is the last will from the previous disconnect — it reads offline however healthy the PLC is. A snapshot without -R will tell you a running controller is dead.

Three shapes, three different investigations:

• Everything dead, buttons included → the application stopped or was stopped. Read the log (§4). This is the case this page is mostly about.
• Buttons dead, MQTT fine → the K-bus. Look for bus is not communicating in the log, and remember the DI process image reads zero while the bus is down, which through a normally-closed loop is the tripped state.
• MQTT dead, buttons fine → the client, the broker, or the network. Nothing in this page applies; start at test-plc-logic.

2. Get onto the controller

SSH, root. The vendor default password is wago on this generation; newer WAGO firmware ships a per-device password printed on the label, and a commissioned box may have had it changed — only WBM as admin can reset it. Never factory-reset to recover a password: that wipes the runtime installation and the boot application with it.

There is no sshpass or plink on the dev machine, so a password cannot be piped in. Use OpenSSH's askpass instead:

printf '#!/bin/sh\necho "<password>"\n' > pw.sh && chmod +x pw.sh
SSH_ASKPASS="$PWD/pw.sh" SSH_ASKPASS_REQUIRE=force DISPLAY=:0 \
  ssh -o StrictHostKeyChecking=no -o PubkeyAuthentication=no root@<ip> 'uptime'
rm -f pw.sh          # do not leave it lying around

SSH_ASKPASS_REQUIRE=force is the part that matters: without it OpenSSH only consults askpass when it has no tty, which depends on how the shell was invoked.

No password at all? Two routes need none, because they go over the runtime connection rather than Linux: the device editor's Files tab browses the controller's filesystem and can download the log, and PLC Shell answers for runtime state (§6).

3. Where the evidence lives

Two traps here, both of which cost a run:

• /var/opt/codesys is a decoy. It holds the licence container, the object database and the audit logs, and it is where the configuration says the log goes — but the runtime's live state is under /home/codesyscontrol/, on a different mount. find / -xdev -name 'codesyscontrol*.log*' returns nothing and looks like a definitive answer. Search without -xdev.
• /var/log is RAM. Anything you want from the previous run is not there. dmesg is equally gone, so an OOM kill cannot be confirmed after a reboot — which is a reason to read the CODESYS log rather than hunt for one.

Also: strings is not on PATH in this Git Bash. strings f | grep x silently produces nothing, which reads exactly like "the file contains no such text". Extract printable runs with Python instead.

4. Read the log — but fix the clock first

Assume the controller's clock is wrong. Neither site controller runs NTP: on 7 Sep 2026 Bijbouw's clock read January 2023 and Stal's read June 2026. Timestamps are internally consistent, so the log is perfectly usable once anchored:

1. date on the controller, next to the real time, gives the offset.
2. Cross-check it against an event you can date independently — the mtime of PlcLogic/<app>/<app>.app against the download you remember making, or an Application ... loaded via [Download] line against a commit.

Anchored to about ±15 minutes, which is ample. Do not skip this: an unanchored log makes a crash from an hour ago look like ancient history.

Then read it:

grep -inE "exception|fatal|abort|watchdog|out of memory" codesyscontrol.log

Logger.0.Filter=0x0000000F (info, warning, error, exception) is the default and lives in CODESYSControl_User.cfg so logsetfilter can change it. Check it before concluding that a silent log means a clean run — but note that at 0x0F exceptions are recorded, so a genuinely silent log means the application did not fault.

5. ProcessorLoadWatchdog: the one that stops a whole building

WAGO arms this in the runtime configuration, and it is easy to be unaware of:

[CmpSchedule]
ProcessorLoad.Enable=1
ProcessorLoad.Maximum=95
ProcessorLoad.Interval=5000

Every 5 seconds it samples PLC load; one sample above 95% raises an exception against App=[all], Task=[all]. It stops every task in every application and leaves them stopped — lights, pushbuttons, covers, MQTT, all of it — until someone restarts the PLC. There is no automatic recovery, and because it acts on a 5-second average it needs only a burst, not a sustained overload.

It matters here because a Freewheeling task consumes every spare cycle by definition, so a controller with one sits permanently near the ceiling.