When a function does not work
Find your symptom below. Each one lists the likely causes and the fix. Start with Find out what happened: it tells you where LEA Platform shows what a function did.
Find out what happened
| Where | What you see |
|---|---|
| Settings → Functions, the function's card, Recent runs | Every run: Status (OK, ERROR, TIMEOUT, PENDING), Trigger, Delivered (what was sent to the host) and the Error. Switch on Show test runs to include tests |
| The same card, Used by rules | Which rules run the function, and how ("on fail of validation · record only") |
| The result (click a row on Execution, or Host · rule in Recent runs) | Under Last check or Last remedy: the Artifact box with status, summary and error. Pending remedy (on the full result page) shows work waiting for the host |
| Test run on the card | The Error, the stderr (last 4 KB of error output), the Artifact and the Event sent |
| The notification bell | Function failed and Function upload awaiting approval |
| Settings → Functions, Runtime (bottom) | Whether functions are enabled on this server, Python version, tools, limits |
| Audit | Who uploaded, approved, tested or changed secrets, and when |
From a terminal: lea functions show NAME, lea functions runs NAME -f, lea functions runtime.
The function is not in the rule editor list
- It was uploaded after you opened the rule editor. Reload the page.
- It is not uploaded on this server. Check Settings → Functions. If it has no card, upload it.
- It waits for approval. A staged upload is not usable until another administrator approves it. The card (if an older version exists) shows UPLOAD #N PENDING; a brand-new function has no card yet. See The upload waits for approval.
- It is already attached to this rule. Each function can be attached once per rule. Add function is disabled when every available function is already attached.
- You are not an administrator. Only administrators can edit rules and functions.
The card says "is not uploaded" or "Unavailable"
- Function "name" is not uploaded. Upload it in Settings → Functions (or lea functions push DIR) before saving. The rule refers to a function that does not exist here, for example a rule imported from another site or a function that was deleted. Upload the function with exactly that name, or remove the hook (bin icon) and save.
- Unavailable with a reason, or the card shows UNAVAILABLE: the function has no code stored on this server (this happens to functions from older LEA Platform releases). Upload the function again; it becomes AVAILABLE. Until then, its runs fail.
A "Then on the host" option is greyed out
| Greyed-out option | Reason | Fix |
|---|---|---|
| Run the remediation with the output and Let the function pick an action | Another hook on this rule already sends work to the host. Only one hook per rule may | Set the other hook to Record only, or remove it |
| Run the remediation with the output | Run when is not Verdict fails | Set Run when to Verdict fails |
| Run the remediation with the output | remediations and actions is ticked | Keep only validations ticked |
| Run the remediation with the output | The rule has no remediation | Add one in the Remediations section |
| Let the function pick an action | The rule has no actions | Add actions under Remediations → Actions (Add action) |
Changing Run when away from Verdict fails switches a hook that used Run the remediation with the output back to Record only.
"Confirm it is you" keeps appearing, or you get 403
- The 10 minutes passed. The confirmation lasts 10 minutes per browser session. Confirm again.
- Single sign-on: you did not repeat the action. After Re-authenticate you come back to the page, but the action is not repeated for you. Do it again; it now goes through.
- "The identity provider did not ask you to sign in again." Your identity provider ignored the request to sign in again. Try again; if it persists, your identity provider administrator must allow forced re-authentication for LEA Platform, or use a passkey.
- "Your identity provider cannot confirm a fresh sign-in." or no Re-authenticate button: your identity provider (for example Google) does not report when you signed in. Add a passkey and use it.
- "You signed in as a different identity." You signed in at the identity provider as another user. Sign in as the same person as in the console.
- "Your identity provider is not available for re-authentication." The identity provider cannot be reached. Use a personal access token with Manage functions from the command line meanwhile.
- Different browser tab or new sign-in. The confirmation belongs to one session. A new sign-in needs a new confirmation.
- Wrong password. Failed confirmations count towards locking your account. If it is locked, ask another administrator.
From the lea command line or CI:
step_up_requiredorfunctions_scope_required: you are using the normallea loginsign-in, or a token without Manage functions. Create a token with Manage functions in Settings → AI Assistant (Create Token) and use it withLEA_TOKEN=...orlea login --token -. See Uploads, approvals and signatures.- Still 403 with the right token: the token expired or was revoked, its owner is no longer an administrator, or the owner has not signed in to the console for 90 days.
The upload is refused
The report's red box or the error message names the reason:
| Message | Meaning and fix |
|---|---|
bundle exceeds 10 MB | The packed upload is too large. Remove test data, caches and unused libraries |
bundle unpacks to more than 64 MB / bundle has more than 10000 files | Unpacked too large or too many files. Same fix |
path ... escapes the function folder, invalid path ... | The archive contains absolute paths or ... Create the archive from inside the parent folder: tar czf my_fn.tgz my_fn |
... must be a regular file (links and devices are refused) | The folder contains a symbolic link. Replace it with a real copy of the file |
... is encrypted | A password-protected zip. Upload an unprotected archive |
... appears twice in the bundle, size does not match the archive header, not a valid .tgz, not a valid .zip | The archive is damaged. Create it again |
send the function as a .tgz, a .zip or a multipart folder upload | The file is not a .tgz or .zip. Pack it again |
the bundle contains no files | The folder or archive is empty |
function.json not found at the bundle root | function.json is missing or one folder too deep. In the console, Create function.json writes one for a folder upload |
function.json is not valid JSON | A syntax error (often a missing comma or a trailing comma). The message gives the position |
name must match ^[a-z0-9_]{1,64}$ and equal the function name | The name in function.json differs from the Function name of the upload (or from the folder name), or has capital letters, - or spaces. Use lowercase letters, digits and _, the same everywhere |
name ... is reserved | policy, uploads, runtime, starters, manifest and test cannot be used |
entrypoint must be a non-empty array of strings | Write it as a list: ["python3", "main.py"] |
timeout must be an integer between 1 and 600 | A whole number of seconds, without quotes |
secrets must be an array of allowed environment variable names (or env ...) | A name is invalid or reserved (for example PATH, or starting with LD_, PYTHON or LEA_). Rename it |
each arg needs a name and a type of string, number or boolean | Fix the args list |
ai_settings must be a boolean | Use true or false without quotes |
a signature is required ... | The policy requires signatures. Sign with lea functions sign and add the .sig |
signed with an unknown key ... | Add that public key under Policy → Trusted signing keys, or sign with a trusted key |
bad signature: the content was modified or signed with another key | A file changed after signing, or the browser folder upload dropped file permissions. Sign again, and upload the archive or use lea functions push |
The upload waits for approval
The policy Require approval by a second administrator is on. The upload is staged, and the old version (if any) keeps running.
- Another administrator must open Review in Waiting for approval on Settings → Functions and click Approve and activate. You cannot approve your own upload, not even with a token.
- If you uploaded again meanwhile, only the newest upload waits; the older one is superseded.
- The approval is refused with
stored bundle does not match its digest(upload it again) orthe upload no longer passes the policy(the signature rules changed since; sign again with a trusted key and upload again).
The function never runs
No row appears in Recent runs after the rule ran on a host. Check in this order:
- The hook does not match the result.
- Run when is Verdict fails but the result passed (or the reverse). Rules without expectations have no verdict: only Every result runs on them.
- for results of has only remediations and actions, but the result is a normal check (or the reverse).
- A condition under Only when the output matches does not match. All conditions must match. Test your regular
expression against the real output shown on the result. Remember
(?m)for^and$on several lines.
- The hook was not saved. Open the function card: is the rule under Used by rules? If you clicked Cancel in the rollout dialog, the hook was not saved.
- The rule does not run on the host. The rule must be active and assigned to the host through a group. The result page says "This rule is not currently assigned to the host through any active group." A host in Probation does not run hooks of rules with secret parameters until it is confirmed.
- The host does not report. Check the host's last check-in on its page. No results, no functions.
- Nothing new to do. If the host sends exactly the same output as before and the verdict did not change, the function does not run again; the earlier artifact is reused and appears on the new result. This is normal.
- The hourly limit. A function runs at most 12 times per hour for the same host, rule and function. Wait, or ask your operator to raise the limit. Test runs do not count.
- Functions are disabled on this server. See Functions are disabled on the server.
The function failed, timed out or returned invalid output
Look at the Error in Recent runs or in the Artifact box, and for details run a Test run on the same result: it shows stderr and the Event sent.
| Error | Cause | Fix |
|---|---|---|
exit N: ... | The code stopped with an error. The text after it is the end of the error output (for example a Python traceback) | Fix the code. Run it locally with the event from the test run |
timed out after Ns | The code ran longer than its timeout | Find the slow call (often a network call without a timeout). Give every network call a timeout, or raise timeout (max 600) |
output is not valid JSON, empty output, output is not a JSON object | Something else was printed to standard output, or nothing | Print exactly one JSON object. Write logs to standard error. In shell, send curl output to /dev/null |
output exceeds 256 KB | The answer is too large | Return less (cut long text) |
invalid output: action must be null, "remedy" or an action name | action is not a valid name | Use an action name from the rule (lowercase, digits, _), "remedy" or null |
invalid output: files must map names matching ... | A file name contains /, starts with ., is too long, is lea_artifact.json, or the content is not text | Use simple names like host.crt and text content |
could not start entrypoint | The program does not exist on the server, or a script is started directly and is not executable | Use ["python3", "main.py"] or ["sh", "main.sh"] |
secret NAME cannot be decrypted, re-enter it | The secret was encrypted with a key that no longer exists (the card shows RE-ENTER) | Set the secret again |
function not available: ... | The function has no stored code | Upload it again |
runner unavailable, runner restarted during the run | The LEA function runner was stopped or restarted | Ask your operator to check the function runner service. The function runs again with the next result |
function was updated while starting, it runs again on the next failure | A new version was activated at that moment | Nothing to do |
After a failure, administrators get a Function failed notification (marked more serious after 3 failures of the same function within an hour). The function is tried again when the host sends the same output again, after 1 minute, then 2, 4, 8 minutes and so on, up to 1 hour. When the output changes, it runs at once.
Nothing happened on the host
The function ran (OK), but the host did not get the remediation or action.
- It is not time yet. Work for the host is delivered on the host's next check-in after the function finished (every 30 seconds by default). Use Speed up check-ins on the host page if you are waiting. The function card shows remedy pending or action pending under Delivered until then.
- The host did not check in within 1 hour. Held work expires after 1 hour. The next failing check starts again.
- The hook is Record only. Nothing is sent by design. Check Used by rules on the function card ("record only").
- The function chose no action. With Let the function pick an action, an answer without
action(or"action": null) sends nothing. Read the artifact's summary. Note: the Delivered column also shows action pending in this case, although nothing will be sent. - The action name is not in the rule for this host. The function named an action the rule does not have, or the
action has no command for the host's operating system. Compare the artifact's
actionwith the rule's actions and their system filters. - The host still runs an older version of the rule. If you added the actions or remediation with New hosts only in the rollout dialog, hosts that already ran the rule keep the old definition without them. Promote those hosts (see Rule library and updates).
- Another hook delivers. Only one hook per rule sends work to the host.
- The function failed. With Let the function pick an action, a failed run sends nothing. With Run the
remediation with the output, the remediation is still sent, with the error in
lea_artifact.jsonand without the function's files; a remediation that checks for its file then stops and is reported as REMEDY FAILED. - Rate limited or busy. When the hourly limit is reached or another run of the same function for the same host and
rule is still going, nothing new is computed. With Run the remediation with the output, the remediation is sent
with that error in
lea_artifact.json.
Secrets are empty in a test run
- You tested a staged upload. Uploads that wait for approval (or are not yet active) run their test runs without secrets and without the AI key. The test run shows NO SECRETS. Test again after approval, from the function card.
- The secret is NOT SET on the function card. Set it.
- The secret is marked NOT DECLARED. The current
function.jsonno longer lists it, so it is not passed. Add it back tosecretsand upload again. - The name differs. Secret names are case-sensitive:
CA_URLandca_urlare different.
You cannot delete a function
The delete icon is disabled ("Detach it from its rules first") while a rule uses the function. The command line says
... is used by rules: ...; detach it first.
- On the function's card, look at Used by rules.
- For each rule: open it, Edit, Functions, remove the hook with the bin icon, Save rule.
- Delete the function. Its secrets are deleted with it.
Functions are disabled on the server
The Runtime section on Settings → Functions shows why execution is unavailable. Functions disabled by an administrator means the Enable functions switch is off under Policy → Execution; an administrator can turn it on again. An executor unavailable or isolation self-check error means the separate executor is stopped or could see protected paths or resolve LEA internal service names. Queued work waits and recovers when the executor is ready. Give that message to your operator. See Install and upgrade. WARNING: metadata block inactive: … means this host could not install or verify metadata blocking. Functions keep running; your operator may apply the optional host rules.
Related messages:
- "The function runner has not reported its runtime yet. Start the function_runner service." or "The function runner has not published its encryption key yet. Start the function_runner service to manage secrets.": the function runner service is not running. Ask your operator to start it.
secrets key not available, start function_runnerwhen setting a secret: same cause.
Saving the rule fails
The rule editor shows the server's reason:
| Message (part) | Fix |
|---|---|
function ... is not uploaded | Upload the function, or remove the hook |
... is already attached to this rule | Remove the duplicate hook |
only one hook per rule can deliver to the host | Set all but one hook to Record only |
deliver remedy runs the remedy of a failed validation, so it needs when fail and on validation | Set Run when to Verdict fails and tick only validations |
deliver remedy needs a remedy in the rule definition | Add a remediation, or choose another option |
deliver action needs actions in the rule definition | Add actions, or choose another option |
functions must be a list of at most 8 hooks | Remove hooks |
... declares no argument ..., argument ... must be a ... | The function's arguments changed. Clear and set the argument fields again |
match ... does not compile | Fix the regular expression in Only when the output matches |
Collect this before asking for help
- The function name and the Digest from its card.
- The rule name, and a screenshot or copy of its Functions section (all fields of the hook).
- One host name where it should have worked, and the time.
- The Recent runs row for that host: Status, Trigger, Delivered, Error.
- A Test run on that host's result: the Error, stderr and Event sent (remove anything sensitive).
- The result page of that host and rule: the Last check output and the Artifact box.
- The host's last check-in time (host page).
- The Runtime section: Python version and any red message.
- The output of
lea functions show NAMEandlea functions runtime, if you use the command line. - Relevant Audit entries (uploads, approvals, secret changes).
Related
- What functions are
- Writing function code
- Uploads, approvals and signatures
- Troubleshooting
- Functions reference: delivery and timing