st.save() error: only length-1 arrays can be converted to Python scalars

[katimbach]

Hi,

Thank you for this awesome tool! I am currently trying to use it to process my own samples with 3-Assembling-HEST-Data.ipynb in the tutorial, but when I get to the point of saving my HESTData there is an error I can't figure out. I know the read function is working well, because st.save_spatial_plot looks good:

Briefly, this is what I am runnning:

from hest import VisiumReader
import os

tissue = "kidney"
sample = "CCRCC4"
base_dir = "/mnt/disks/kimbach-spatbinn/data"
sample_dir = os.path.join(base_dir, tissue, sample)

processed_dir = os.path.join(sample_dir, 'processed')
if not os.path.exists(processed_dir):
    os.makedirs(processed_dir)

fullres_img_path = os.path.join(base_dir, tissue, sample + '.tif')
bc_matrix_path = os.path.join(sample_dir, 'filtered_feature_bc_matrix.h5')
spatial_coord_path = os.path.join(sample_dir,'spatial')

st = VisiumReader().read(
    fullres_img_path, # path to a full res image
    bc_matrix_path, # path to filtered_feature_bc_matrix.h5
    spatial_coord_path=spatial_coord_path # path to a space ranger spatial/ folder 
)

#Save the HESTData
save_dir = processed_dir
st.save(path= save_dir)

And this is the error I am getting:

TypeError Traceback (most recent call last)
Cell In[6], line 3
1 #Save the HESTData
2 save_dir = processed_dir
----> 3 st.save(path= save_dir)

File ~/HEST/src/hest/HESTData.py:196, in HESTData.save(self, path, save_img, pyramidal, bigtiff, plot_pxl_size, **kwargs)
192 self.save_tissue_vis(path, 'tissue_seg')
195 if save_img:
--> 196 tiff_save(img, os.path.join(path, ALIGNED_HE_FILENAME), self.pixel_size, pyramidal=pyramidal, bigtiff=bigtiff)

File ~/HEST/src/hest/utils.py:884, in tiff_save(img, save_path, pixel_size, pyramidal, bigtiff)
882 print("Couldn't import pyvips, verify that libvips is installed on your system")
883 print('saving to pyramidal tiff... can be slow')
--> 884 pyvips_img = pyvips.Image.new_from_array(img)
886 # save in the generic tiff format readable by both openslide and QuPath
887 # Note: had to change the compression from 'deflate' to 'lzw' because of a reading incompatibility with CuImage/OpenSlide
888 # when upgrading to vips 8.13 (necessary for Valis)
889 pyvips_img.tiffsave(
890 save_path,
891 bigtiff=bigtiff,
(...)
898 xres=1. / (pixel_size * 1e-4),
899 yres=1. / (pixel_size * 1e-4))

File ~/miniconda3/envs/hest/lib/python3.9/site-packages/pyvips/vimage.py:303, in Image.new_from_array(array, scale, offset)
301 for y in range(0, height):
302 for x in range(0, width):
--> 303 a[x + y * width] = array[y][x]
305 vi = vips_lib.vips_image_new_matrix_from_array(width, height, a, n)
306 if vi == ffi.NULL:

TypeError: only length-1 arrays can be converted to Python scalars

[pauldoucet]

Hi @katimbach

After reading the sample with st = VisiumReader().read(...), can you print the value of st.wsi.numpy().shape please?

[katimbach]

Hi @pauldoucet,

Thanks so much for your quick response on this!! The output of st.wsi.numpy().shape is (9282, 9464, 3).

For your reference, I am reading a Visium sample with corresponding 10X magnification WSI.

[pauldoucet]

Can you run this line on your machine please:
pyvips.Image.new_from_array(np.zeros((9282, 9464, 3)))

Also can you run:
pip freeze | grep pyvips

and

vips -version

This works fine on my machine, if it throws an exception, please update pyvips on your system. (I'm using pyvips=2.2.3)

If it still doesn't work, update the libvips version on your system I'm using libvips==8.16.0 but >=8.4.5 should be fine

[katimbach]

Hi,

Running pyvips.Image.new_from_array(np.zeros((9282, 9464, 3))) returns

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bscuser/miniconda3/envs/hest/lib/python3.9/site-packages/pyvips/vimage.py", line 303, in new_from_array
    a[x + y * width] = array[y][x]
TypeError: only length-1 arrays can be converted to Python scalars

The vips version is vips-8.11.4.

Running pip freeze | grep pyvips gives:
pyvips @ file:///home/conda/feedstock_root/build_artifacts/pyvips_1624887326397/work

Running conda list in my env, I see that my pyvips=2.2.3.

At first glance, attempting to update libvips seems to cause issues with hest:

## Package Plan ##

  environment location: /home/bscuser/miniconda3/envs/hest

  added / updated specs:
    - conda-forge::libvips


The following packages will be REMOVED:

  hest-1.1.1-pypi_0

The following packages will be SUPERSEDED by a higher-priority channel:

  pyvips                     pypi/pypi::pyvips-2.2.3-pypi_0 --> conda-forge/linux-64::pyvips-2.1.15-py39h3811e60_0 


Proceed ([y]/n)?

[pauldoucet]

@katimbach,
I couldn't reproduce the error in a fresh environment with pyvips 2.2.3 and libvips 8.11.4 (Ubuntu 22.04). I suspect you might have multiple versions of libvips installed on your system, and that pyvips is not using the correct one.

Could you confirm which libvips version pyvips is using with this snippet:

import pyvips # pyvips 2.2.3
import numpy as np

print(f"libvips version: {pyvips.version(0)}.{pyvips.version(1)}.{pyvips.version(2)}") #libvips version: 8.11.4

a = pyvips.Image.new_from_array(np.zeros((9282, 9464, 3))) #<pyvips.Image 9464x9282 double, 3 bands, srgb>

If this is still throwing an exception, please reinstall hest and libvips on a fresh conda environment:

conda create -n hest python=3.10
conda activate hest
git clone git@github.com:mahmoodlab/HEST.git
cd HEST
pip install -e .
conda install -c conda-forge libvips=8.11.4