H.264 encode returns empty frames on vepu540c: `hal_h264e_vepu540c` never reads back the hardware status register

[sormy]

On vepu540c (RK3528-class, DT reports rockchip,rk3528a), H.264 encode produces zero-length
frames at every resolution and rate-control mode. H.265 and MJPEG on the same encoder are fine.
MPP develop @ 8f922ed, unmodified; kernel 6.1.115 vendor, mpp_rkvenc2.

$ mpi_enc_test -t 7 -w 1920 -h 1080 -n 30 -o /tmp/h264.bin
hal_h264e_vepu540c_status_check enc not done hw_status: 0x00000000
chn 0 encoded frame 29   size 0

The hardware is finishing the work: across a 10-frame run the rkvenc IRQ in /proc/interrupts
increments exactly 10 times, same as a working H.265 run. Only the status read is wrong.

hal_h264e_vepu540c_status_check() tests reg_ctl.common.int_sta.enc_done_sta, but
hal_h264e_vepu540c_start() only ever writes the control block — its single MPP_DEV_REG_RD
covers reg_st. So int_sta still holds the zero MPP wrote, and the check fails unconditionally.
hal_h265e_vepu540c_start() reads that word explicitly from VEPU540C_REG_BASE_HW_STATUS first,
which is why H.265 works.

Adding the same read-back fixes it — H.264 then encodes at 116 / 55 / 14.4 / 3.6 fps for
720p / 1080p / 4K / 8K, ffprobe-verified and decodable, with H.265 unaffected:

--- a/mpp/hal/rkenc/h264e/hal_h264e_vepu540c.c
+++ b/mpp/hal/rkenc/h264e/hal_h264e_vepu540c.c
@@ -1600,6 +1600,18 @@ static MPP_RET hal_h264e_vepu540c_start(void *hal, HalEncTask *task)
             break;
         }
 
+        /* int_sta lives in the control block, which is only ever written above; read it
+           back the way hal_h265e_vepu540c does, or enc_done_sta stays the 0 we wrote. */
+        rd_cfg.reg = &ctx->regs_set->reg_ctl.common.int_sta;
+        rd_cfg.size = sizeof(RK_U32);
+        rd_cfg.offset = VEPU540C_REG_BASE_HW_STATUS;
+
+        ret = mpp_dev_ioctl(ctx->dev, MPP_DEV_REG_RD, &rd_cfg);
+        if (ret) {
+            mpp_err_f("set register read failed %d\n", ret);
+            break;
+        }
+
         if (hal_dbg_flag_en(ctx->dbg_ctx, HAL_DBG_GET_REG)) {
             RK_S32 ret_dbg = 0;
             vepu540c_get_dbg_regs(ctx->dev, ctx->regs_set, reg_s3, ret_dbg);

Is reading int_sta back the intended fix, or should status_check() read from reg_st instead? I
inferred the layout from the H.265 path rather than documentation. Happy to open a PR instead.

[Liaoyyyy]

Thanks for the report and the detailed analysis!
We've confirmed and fixed the issue internally — it will be synced to the GitHub repo with the next upstream update.