Skip to content

Conversation

@oshoval
Copy link
Collaborator

@oshoval oshoval commented Nov 19, 2025

What this PR does / why we need it:

  • Add standard log levels.
  • Remove repetitive reconciliation logs, those spammed badly and were not helpful.
    For example from a cluster that ran for a while:
❌ log.Print("reconciling NetworkAddonsConfig") - 2,585 occurrences
❌ log.Print("starting render phase") - 2,585 occurrences
❌ log.Print("starting rendering objects to delete phase") - ~2,585 occurrences
❌ log.Print("update was successful") - 98,230 occurrences
❌ log.Print("Successfully updated status conditions") - 5,193 occurrences

Special notes for your reviewer:

Release note:

None

Remove repetitive reconciliation logs.

Signed-off-by: Or Shoval <[email protected]>
@kubevirt-bot kubevirt-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Nov 19, 2025
@kubevirt-bot
Copy link
Collaborator

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@kubevirt-bot kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/XS size/L and removed size/XS labels Nov 19, 2025
@oshoval oshoval changed the title logs: Reduce logs spamming WIP logs: Reduce logs spamming Nov 19, 2025
@oshoval oshoval marked this pull request as ready for review November 19, 2025 14:14
@oshoval
Copy link
Collaborator Author

oshoval commented Nov 19, 2025

Some logs that might worth to investigate on follow-ups.
Easier to spot them now, once it is cleaner.
there might be more, just took those 3 as example.

  1. seems that need to change 2nd container port name (KSD)
{"level":"info","ts":"2025-11-19T14:34:41Z","msg":"spec.template.spec.containers[1].ports[0]: duplicate port name \"healthport\" with spec.template.spec.containers[0].ports[1], services and probes that select ports by name will use spec.template.spec.containers[0].ports[1]"}

Note that i dont see it on KCLI, just on k8s.

  1. Understand what are those 14 objs (actually it doesnt remove nothing at the end, maybe those are secrets and such that are later ignored)
{"level":"info","ts":"2025-11-19T14:37:42Z","logger":"render","msg":"object removal render phase done","objectsToRemove":14}

those are the oldKNMStateObjects,oldIPAMControllerPasstObjects
causing this print to be periodic, we can fix it on a follow-up either by dropping them in case 3 versions passed, or by ignoring the count if its only them.
Even easier, i am raising it to V(1).

{"level":"info","ts":"2025-11-19T14:37:42Z","msg":"unknown field \"spec.template.spec.securityContext.readOnlyRootFilesystem\""}

should be moved to the container (but need to be tested that it still works afterwards)
we should make it fatal instead warning (maybe on the first one as well)

{"level":"info","ts":"2025-11-20T10:57:51Z","msg":"spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[1].preference.matchExpressions[0].key: node-role.kubernetes.io/master is use \"node-role.kubernetes.io/control-plane\" instead"}

we do have both, maybe we can remove the deprecated one, in case its not needed anymore for BWC

2,3 repeat each reconcile before this PR

cc @RamLavi fyi please

@oshoval
Copy link
Collaborator Author

oshoval commented Nov 19, 2025

/test pull-e2e-cluster-network-addons-operator-macvtap-cni-functests

Copy link
Collaborator

@RamLavi RamLavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, I think it's a great idea to introduce leveled logs.
Could you have it configurable on the operator via some flag? This way, the could control how much logging they see.

@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

Hey, I think it's a great idea to introduce leveled logs. Could you have it configurable on the operator via some flag? This way, the could control how much logging they see.

done

@oshoval oshoval changed the title WIP logs: Reduce logs spamming logs: Reduce logs spamming and introduce levels Nov 20, 2025
@kubevirt-bot kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 20, 2025
@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

/hold need to fix something

@kubevirt-bot kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 20, 2025
Allows to use levels.

Signed-off-by: Or Shoval <[email protected]>
@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

/hold cancel

@kubevirt-bot kubevirt-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 20, 2025
@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

/retest

@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

updated findings to #2491 (comment)

the 3rd issue is a real bug, the first is nice to fix as well, but self healing

In order to enable set CNAO_LOG_LEVEL=-1.

Signed-off-by: Or Shoval <[email protected]>
@sonarqubecloud
Copy link

@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

changes: Reduce verbosity of repetitive logs

@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

unrelated to this PR
on Kcli i see this, fyi @RamLavi
maybe i am doing something wrong, but linux-bridge is installed on OpenShift

{"level":"info","ts":"2025-11-20T10:57:50Z","msg":"would violate PodSecurity \"restricted:latest\": privileged (container \"cni-plugins\" must not set securityContext.privileged=true), seLinuxOptions (pod set forbidden securityContext.seLinuxOptions: type \"spc_t\"), allowPrivilegeEscalation != false (container \"cni-plugins\" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container \"cni-plugins\" must set securityContext.capabilities.drop=[\"ALL\"]), restricted volume types (volume \"cnibin\" uses restricted volume type \"hostPath\"), runAsNonRoot != true (pod or container \"cni-plugins\" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container \"cni-plugins\" must set securityContext.seccompProfile.type to \"RuntimeDefault\" or \"Localhost\")"}

@oshoval
Copy link
Collaborator Author

oshoval commented Nov 20, 2025

point 3 of the above issues is fixed by #2497

Copy link
Collaborator

@RamLavi RamLavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 20, 2025
@oshoval
Copy link
Collaborator Author

oshoval commented Nov 23, 2025

/approve

@kubevirt-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: oshoval

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 23, 2025
@kubevirt-bot kubevirt-bot merged commit be55d1e into kubevirt:main Nov 23, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants