WeakFixation

Overview

This repository is the implementation of 'Weakly Supervised Visual Saliency Prediction (TIP2022)'.

Qiuxia Lai, Tianfei Zhou, Salman Khan, Hanqiu Sun, Jianbing Shen, Ling Shao.

Requirements

Create an anaconda environment:

$ conda env create -f environment.yml

Activate the environment:

$ source activate torch36
$ <run_python_command> # see the examples below

Results Download

Prediction results on MIT300, MIT1003, PASCAL-S, SALITON-Test, TORONTO, and DUT-OMRON can be downloaded from:

Google Drive: https://drive.google.com/file/d/1CWWv79RYwh1tRY82VtsjZ1N4KxyccTa_/view?usp=sharing

Baidu Disk: https://pan.baidu.com/s/1HfZNfNAsKqzJRAbX4WU7eA (password:s216)

Our evaluation code is adapted from this matlab tool.

Datasets Preparation

For training

We use MS-COCO 2014 - train for training, and a subset of MS-COCO - val (i.e., the val of SALICON) for evaluation.

Images available at the official website.

Bounding boxes: Baidu Disk Link (train) (password:5ecm) and Baidu Disk Link (eval) (password:qdrg). OR generated using EdgeBox.

Saliency prior maps: Baidu Disk Link (train) (password:u8h2) and Baidu Disk Link (val) (password:qxxe). OR generated using the official matlab code of Dynamic visual attention: Searching for coding length increments, NeurIPS 2008.

For testing

Taking MIT1003 as an example. You may download the dataset along with the bounding boxes from this Baidu Disk Link (password:ten8) OR Google Drive for a fast try.

Dataset arrangement

The datasets are arranged as:

    DataSets
    |----MS_COCO
         |---train2014
         |---val2014
         |---train2014_eb500
         |---val2014_eb500
         |---train2014_nips08
         |---val2014_nips08
    |----SALICON
         |---images
            |---train
            |---val
            |---test
         |---fixations
            |---train
            |---val
            |---test
         |---maps
            |---train
            |---val
         |---eb500
            |---train
            |---val
    |----MIT1003
         |---ALLSTIMULI
             |--xxx.jpeg
             |--xxx.jpeg
             |--...
         |---ALLFIXATIONMAPS
             |--...
         |---ALLFIXATIONS
             |--...
         |---eb500
             |--xxx.mat
             |--...

Testing

Download weight model_best.pt from:

Google Drive: https://drive.google.com/file/d/1KxyXNWo_mxPkRo1sf2jHFMB_Jxzc6msY/view?usp=sharing

Baidu Disk: https://pan.baidu.com/s/1Mn7U3UTKOVUW7w6WC5w65w password:bgft

Configurations

1. Set the base_path in config.py be the parent folder of DataSets.
2. Put the downloaded model_best.pt in <code_path>/WF/Models/best/.

Prediction

Run

python main.py --phase test --model_name best --bestname model_best.pt --batch-size 2

The saliency prediction results will be saved in <code_path>/WF/Preds/MIT1003/<model_name>_multiscale/.

Please evaluate the prediction results using the above mentioned matlab tool.

Training

Coming soon.

Citation

If you find this repository useful, please consider citing the following reference.

@ARTICLE{lai2022weakly,
    title={Weakly supervised visual saliency prediction},
    author={Qiuxia Lai and Tianfei Zhou and Salman Khan and Hanqiu Sun and Jianbing Shen and Ling Shao},
    journal={IEEE Trans. on Image Processing},
    year={2022}
}

Contact

Qiuxia Lai: ashleylqxatgmail.com | qxlaiatcuc.edu.cn