Skip to content

Documentation

When a function does not work

Find your symptom below. Each one lists the likely causes and the fix. Start with Find out what happened: it tells you where LEA Platform shows what a function did.

Find out what happened

WhereWhat you see
Settings → Functions, the function's card, Recent runsEvery run: Status (OK, ERROR, TIMEOUT, PENDING), Trigger, Delivered (what was sent to the host) and the Error. Switch on Show test runs to include tests
The same card, Used by rulesWhich rules run the function, and how ("on fail of validation · record only")
The result (click a row on Execution, or Host · rule in Recent runs)Under Last check or Last remedy: the Artifact box with status, summary and error. Pending remedy (on the full result page) shows work waiting for the host
Test run on the cardThe Error, the stderr (last 4 KB of error output), the Artifact and the Event sent
The notification bellFunction failed and Function upload awaiting approval
Settings → Functions, Runtime (bottom)Whether functions are enabled on this server, Python version, tools, limits
AuditWho uploaded, approved, tested or changed secrets, and when

From a terminal: lea functions show NAME, lea functions runs NAME -f, lea functions runtime.

The function is not in the rule editor list

  • It was uploaded after you opened the rule editor. Reload the page.
  • It is not uploaded on this server. Check Settings → Functions. If it has no card, upload it.
  • It waits for approval. A staged upload is not usable until another administrator approves it. The card (if an older version exists) shows UPLOAD #N PENDING; a brand-new function has no card yet. See The upload waits for approval.
  • It is already attached to this rule. Each function can be attached once per rule. Add function is disabled when every available function is already attached.
  • You are not an administrator. Only administrators can edit rules and functions.

The card says "is not uploaded" or "Unavailable"

  • Function "name" is not uploaded. Upload it in Settings → Functions (or lea functions push DIR) before saving. The rule refers to a function that does not exist here, for example a rule imported from another site or a function that was deleted. Upload the function with exactly that name, or remove the hook (bin icon) and save.
  • Unavailable with a reason, or the card shows UNAVAILABLE: the function has no code stored on this server (this happens to functions from older LEA Platform releases). Upload the function again; it becomes AVAILABLE. Until then, its runs fail.

A "Then on the host" option is greyed out

Greyed-out optionReasonFix
Run the remediation with the output and Let the function pick an actionAnother hook on this rule already sends work to the host. Only one hook per rule maySet the other hook to Record only, or remove it
Run the remediation with the outputRun when is not Verdict failsSet Run when to Verdict fails
Run the remediation with the outputremediations and actions is tickedKeep only validations ticked
Run the remediation with the outputThe rule has no remediationAdd one in the Remediations section
Let the function pick an actionThe rule has no actionsAdd actions under Remediations → Actions (Add action)

Changing Run when away from Verdict fails switches a hook that used Run the remediation with the output back to Record only.

"Confirm it is you" keeps appearing, or you get 403

  • The 10 minutes passed. The confirmation lasts 10 minutes per browser session. Confirm again.
  • Single sign-on: you did not repeat the action. After Re-authenticate you come back to the page, but the action is not repeated for you. Do it again; it now goes through.
  • "The identity provider did not ask you to sign in again." Your identity provider ignored the request to sign in again. Try again; if it persists, your identity provider administrator must allow forced re-authentication for LEA Platform, or use a passkey.
  • "Your identity provider cannot confirm a fresh sign-in." or no Re-authenticate button: your identity provider (for example Google) does not report when you signed in. Add a passkey and use it.
  • "You signed in as a different identity." You signed in at the identity provider as another user. Sign in as the same person as in the console.
  • "Your identity provider is not available for re-authentication." The identity provider cannot be reached. Use a personal access token with Manage functions from the command line meanwhile.
  • Different browser tab or new sign-in. The confirmation belongs to one session. A new sign-in needs a new confirmation.
  • Wrong password. Failed confirmations count towards locking your account. If it is locked, ask another administrator.

From the lea command line or CI:

  • step_up_required or functions_scope_required: you are using the normal lea login sign-in, or a token without Manage functions. Create a token with Manage functions in Settings → AI Assistant (Create Token) and use it with LEA_TOKEN=... or lea login --token -. See Uploads, approvals and signatures.
  • Still 403 with the right token: the token expired or was revoked, its owner is no longer an administrator, or the owner has not signed in to the console for 90 days.

The upload is refused

The report's red box or the error message names the reason:

MessageMeaning and fix
bundle exceeds 10 MBThe packed upload is too large. Remove test data, caches and unused libraries
bundle unpacks to more than 64 MB / bundle has more than 10000 filesUnpacked too large or too many files. Same fix
path ... escapes the function folder, invalid path ...The archive contains absolute paths or ... Create the archive from inside the parent folder: tar czf my_fn.tgz my_fn
... must be a regular file (links and devices are refused)The folder contains a symbolic link. Replace it with a real copy of the file
... is encryptedA password-protected zip. Upload an unprotected archive
... appears twice in the bundle, size does not match the archive header, not a valid .tgz, not a valid .zipThe archive is damaged. Create it again
send the function as a .tgz, a .zip or a multipart folder uploadThe file is not a .tgz or .zip. Pack it again
the bundle contains no filesThe folder or archive is empty
function.json not found at the bundle rootfunction.json is missing or one folder too deep. In the console, Create function.json writes one for a folder upload
function.json is not valid JSONA syntax error (often a missing comma or a trailing comma). The message gives the position
name must match ^[a-z0-9_]{1,64}$ and equal the function nameThe name in function.json differs from the Function name of the upload (or from the folder name), or has capital letters, - or spaces. Use lowercase letters, digits and _, the same everywhere
name ... is reservedpolicy, uploads, runtime, starters, manifest and test cannot be used
entrypoint must be a non-empty array of stringsWrite it as a list: ["python3", "main.py"]
timeout must be an integer between 1 and 600A whole number of seconds, without quotes
secrets must be an array of allowed environment variable names (or env ...)A name is invalid or reserved (for example PATH, or starting with LD_, PYTHON or LEA_). Rename it
each arg needs a name and a type of string, number or booleanFix the args list
ai_settings must be a booleanUse true or false without quotes
a signature is required ...The policy requires signatures. Sign with lea functions sign and add the .sig
signed with an unknown key ...Add that public key under Policy → Trusted signing keys, or sign with a trusted key
bad signature: the content was modified or signed with another keyA file changed after signing, or the browser folder upload dropped file permissions. Sign again, and upload the archive or use lea functions push

The upload waits for approval

The policy Require approval by a second administrator is on. The upload is staged, and the old version (if any) keeps running.

  • Another administrator must open Review in Waiting for approval on Settings → Functions and click Approve and activate. You cannot approve your own upload, not even with a token.
  • If you uploaded again meanwhile, only the newest upload waits; the older one is superseded.
  • The approval is refused with stored bundle does not match its digest (upload it again) or the upload no longer passes the policy (the signature rules changed since; sign again with a trusted key and upload again).

The function never runs

No row appears in Recent runs after the rule ran on a host. Check in this order:

  1. The hook does not match the result.
    • Run when is Verdict fails but the result passed (or the reverse). Rules without expectations have no verdict: only Every result runs on them.
    • for results of has only remediations and actions, but the result is a normal check (or the reverse).
    • A condition under Only when the output matches does not match. All conditions must match. Test your regular expression against the real output shown on the result. Remember (?m) for ^ and $ on several lines.
  2. The hook was not saved. Open the function card: is the rule under Used by rules? If you clicked Cancel in the rollout dialog, the hook was not saved.
  3. The rule does not run on the host. The rule must be active and assigned to the host through a group. The result page says "This rule is not currently assigned to the host through any active group." A host in Probation does not run hooks of rules with secret parameters until it is confirmed.
  4. The host does not report. Check the host's last check-in on its page. No results, no functions.
  5. Nothing new to do. If the host sends exactly the same output as before and the verdict did not change, the function does not run again; the earlier artifact is reused and appears on the new result. This is normal.
  6. The hourly limit. A function runs at most 12 times per hour for the same host, rule and function. Wait, or ask your operator to raise the limit. Test runs do not count.
  7. Functions are disabled on this server. See Functions are disabled on the server.

The function failed, timed out or returned invalid output

Look at the Error in Recent runs or in the Artifact box, and for details run a Test run on the same result: it shows stderr and the Event sent.

ErrorCauseFix
exit N: ...The code stopped with an error. The text after it is the end of the error output (for example a Python traceback)Fix the code. Run it locally with the event from the test run
timed out after NsThe code ran longer than its timeoutFind the slow call (often a network call without a timeout). Give every network call a timeout, or raise timeout (max 600)
output is not valid JSON, empty output, output is not a JSON objectSomething else was printed to standard output, or nothingPrint exactly one JSON object. Write logs to standard error. In shell, send curl output to /dev/null
output exceeds 256 KBThe answer is too largeReturn less (cut long text)
invalid output: action must be null, "remedy" or an action nameaction is not a valid nameUse an action name from the rule (lowercase, digits, _), "remedy" or null
invalid output: files must map names matching ...A file name contains /, starts with ., is too long, is lea_artifact.json, or the content is not textUse simple names like host.crt and text content
could not start entrypointThe program does not exist on the server, or a script is started directly and is not executableUse ["python3", "main.py"] or ["sh", "main.sh"]
secret NAME cannot be decrypted, re-enter itThe secret was encrypted with a key that no longer exists (the card shows RE-ENTER)Set the secret again
function not available: ...The function has no stored codeUpload it again
runner unavailable, runner restarted during the runThe LEA function runner was stopped or restartedAsk your operator to check the function runner service. The function runs again with the next result
function was updated while starting, it runs again on the next failureA new version was activated at that momentNothing to do

After a failure, administrators get a Function failed notification (marked more serious after 3 failures of the same function within an hour). The function is tried again when the host sends the same output again, after 1 minute, then 2, 4, 8 minutes and so on, up to 1 hour. When the output changes, it runs at once.

Nothing happened on the host

The function ran (OK), but the host did not get the remediation or action.

  • It is not time yet. Work for the host is delivered on the host's next check-in after the function finished (every 30 seconds by default). Use Speed up check-ins on the host page if you are waiting. The function card shows remedy pending or action pending under Delivered until then.
  • The host did not check in within 1 hour. Held work expires after 1 hour. The next failing check starts again.
  • The hook is Record only. Nothing is sent by design. Check Used by rules on the function card ("record only").
  • The function chose no action. With Let the function pick an action, an answer without action (or "action": null) sends nothing. Read the artifact's summary. Note: the Delivered column also shows action pending in this case, although nothing will be sent.
  • The action name is not in the rule for this host. The function named an action the rule does not have, or the action has no command for the host's operating system. Compare the artifact's action with the rule's actions and their system filters.
  • The host still runs an older version of the rule. If you added the actions or remediation with New hosts only in the rollout dialog, hosts that already ran the rule keep the old definition without them. Promote those hosts (see Rule library and updates).
  • Another hook delivers. Only one hook per rule sends work to the host.
  • The function failed. With Let the function pick an action, a failed run sends nothing. With Run the remediation with the output, the remediation is still sent, with the error in lea_artifact.json and without the function's files; a remediation that checks for its file then stops and is reported as REMEDY FAILED.
  • Rate limited or busy. When the hourly limit is reached or another run of the same function for the same host and rule is still going, nothing new is computed. With Run the remediation with the output, the remediation is sent with that error in lea_artifact.json.

Secrets are empty in a test run

  • You tested a staged upload. Uploads that wait for approval (or are not yet active) run their test runs without secrets and without the AI key. The test run shows NO SECRETS. Test again after approval, from the function card.
  • The secret is NOT SET on the function card. Set it.
  • The secret is marked NOT DECLARED. The current function.json no longer lists it, so it is not passed. Add it back to secrets and upload again.
  • The name differs. Secret names are case-sensitive: CA_URL and ca_url are different.

You cannot delete a function

The delete icon is disabled ("Detach it from its rules first") while a rule uses the function. The command line says ... is used by rules: ...; detach it first.

  1. On the function's card, look at Used by rules.
  2. For each rule: open it, Edit, Functions, remove the hook with the bin icon, Save rule.
  3. Delete the function. Its secrets are deleted with it.

Functions are disabled on the server

The Runtime section on Settings → Functions shows why execution is unavailable. Functions disabled by an administrator means the Enable functions switch is off under Policy → Execution; an administrator can turn it on again. An executor unavailable or isolation self-check error means the separate executor is stopped or could see protected paths or resolve LEA internal service names. Queued work waits and recovers when the executor is ready. Give that message to your operator. See Install and upgrade. WARNING: metadata block inactive: … means this host could not install or verify metadata blocking. Functions keep running; your operator may apply the optional host rules.

Related messages:

  • "The function runner has not reported its runtime yet. Start the function_runner service." or "The function runner has not published its encryption key yet. Start the function_runner service to manage secrets.": the function runner service is not running. Ask your operator to start it.
  • secrets key not available, start function_runner when setting a secret: same cause.

Saving the rule fails

The rule editor shows the server's reason:

Message (part)Fix
function ... is not uploadedUpload the function, or remove the hook
... is already attached to this ruleRemove the duplicate hook
only one hook per rule can deliver to the hostSet all but one hook to Record only
deliver remedy runs the remedy of a failed validation, so it needs when fail and on validationSet Run when to Verdict fails and tick only validations
deliver remedy needs a remedy in the rule definitionAdd a remediation, or choose another option
deliver action needs actions in the rule definitionAdd actions, or choose another option
functions must be a list of at most 8 hooksRemove hooks
... declares no argument ..., argument ... must be a ...The function's arguments changed. Clear and set the argument fields again
match ... does not compileFix the regular expression in Only when the output matches

Collect this before asking for help

  • The function name and the Digest from its card.
  • The rule name, and a screenshot or copy of its Functions section (all fields of the hook).
  • One host name where it should have worked, and the time.
  • The Recent runs row for that host: Status, Trigger, Delivered, Error.
  • A Test run on that host's result: the Error, stderr and Event sent (remove anything sensitive).
  • The result page of that host and rule: the Last check output and the Artifact box.
  • The host's last check-in time (host page).
  • The Runtime section: Python version and any red message.
  • The output of lea functions show NAME and lea functions runtime, if you use the command line.
  • Relevant Audit entries (uploads, approvals, secret changes).