WeVisOne-VL-2B: Cut Out Objects and Remove Backgrounds Locally (Linux, Windows, Mac)

What you get in one minute#

WeVisOne-VL-2B is a small open vision model from Tencent that takes a picture and a text query such as "person", finds the objects, and draws boxes, masks and soft-edged alpha mattes you can use to remove a background.

Logo: WeChat, the maker's team brand, from the official repository (github.com/Tencent/WeVisOne-VL). The project has no model-specific logo.

One model covers detection, counting, pointing, text reading, segmentation and matting, and it says None when the thing you asked for is not in the picture. After a one-time download of the weights it runs on your own machine, so your pictures stay with you. The honest catch: the project is days old, no minimum hardware is published, and we did not run it ourselves.

At a glance:

• What it does: text query in; boxes, points, counts, text, masks or alpha mattes out.
• Who it is for: people who want local cutouts and object finding without sending pictures to a cloud service.
• What you need: Python 3.10, Git, about 10 GB of free disk, and a GPU if you want it quick.
• Windows and Mac: pip installs work on Windows; on Mac only Apple Silicon works and the official install file needs a change. Details in each step.
• Watch out: the command line tool and the demo load the bigger 4B model unless you ask for 2B.

How it fits together#

You give the model an image and a text query; the vision-language model writes a box plus a special token, and a small pixel decoder turns that into a matte saved as files on your disk.

The pixel decoder has 9.6 million parameters and reuses the language model's own image features. The project says no SAM model is needed to run it. Tencent did use SAM 2 and SAM 3 as helpers to build its training data.

Before you start#

You need Python 3.10, Git and Conda, plus an NVIDIA GPU if you do not want to wait. The README installs with Conda and Python 3.10, so we do the same.

Check what you have:

python3 --version
git --version
conda --version
nvidia-smi

On Windows run python --version, git --version, conda --version and nvidia-smi in PowerShell. On a Mac skip nvidia-smi.

Checklist:

• [ ] About 10 GB free disk (4.9 GB weights plus the Python packages; our estimate)
• [ ] Internet for the first run only (weights come from Hugging Face)
• [ ] Port 8080 free if you start the web demo
• [ ] You have read the privacy notes in "Security and testing"
• [ ] A copy of any existing Conda environment you care about

Tested in October 2026: Linux = not run, Windows = not run, Mac = not run. Every command below was checked against the project's own files, not executed.

Tip: Give Conda its own environment for this project. The official file pins exact versions of PyTorch, Transformers and NumPy, and they will fight with your other projects.

Step 1: Get the code and create the environment#

Clone the repository and make a Python 3.10 environment named wevisone-vl. The commands are from the README.

Linux#

git clone https://github.com/Tencent/WeVisOne-VL.git && cd WeVisOne-VL
conda create -n wevisone-vl python=3.10 -y && conda activate wevisone-vl

Windows (PowerShell)#

git clone https://github.com/Tencent/WeVisOne-VL.git
Set-Location WeVisOne-VL
conda create -n wevisone-vl python=3.10 -y
conda activate wevisone-vl

If conda activate is not found in PowerShell, run conda init powershell once and open a new window.

Mac (Terminal)#

Same commands as Linux:

git clone https://github.com/Tencent/WeVisOne-VL.git && cd WeVisOne-VL
conda create -n wevisone-vl python=3.10 -y && conda activate wevisone-vl

Check it worked:

python --version

Expected: Python 3.10.x, and your prompt starts with (wevisone-vl).

Step 2: Install the packages#

Install the pinned requirements, then the project itself. The README says pip install -r requirements.txt && pip install -v -e .. That works as written on Linux. Windows and Mac need one adjustment each.

Linux#

pip install -r requirements.txt && pip install -v -e .

This pulls torch==2.11.0, transformers==5.12.1, gradio==6.30.0 and, on Linux only, vllm==0.23.0 and triton==3.6.0. Expect several gigabytes of downloads.

Windows (PowerShell)#

Plain pip install torch on Windows gives the CPU-only build. With an NVIDIA GPU, install the CUDA build first, using the version the project pins. The PyTorch download index lists torch 2.11.0+cu128 and torchvision 0.26.0+cu128 for Windows:

pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
pip install -v -e .

The -r requirements.txt line then sees torch==2.11.0 already installed. Without an NVIDIA GPU, skip the first line. The official file marks vLLM and Triton as "not on Windows", so they are skipped for you.

Mac (Terminal)#

There is no official Mac instruction. Two facts from the package index (read October 2026): PyTorch 2.11.0 has Mac wheels for Apple Silicon only, and Triton 3.6.0 has no Mac files at all. So pip install -r requirements.txt should fail on a Mac. The project's pyproject.toml lists the core packages without vLLM and Triton, so install the project itself and add the demo package by hand. This is our reading of the files, not an official method:

pip install -v -e .
pip install gradio==6.30.0

An Intel Mac has no matching PyTorch wheel; use another machine.

Check it worked:

python -c "import torch, transformers; print(torch.__version__, transformers.__version__, torch.cuda.is_available())"

Expected: 2.11.0 5.12.1 followed by True on a working NVIDIA setup (or 2.11.0+cu128 on Windows). False means CPU only; that works but is slow, and on a Mac it is normal.

Potential issue: on Windows, False after the CUDA install means pip resolved the CPU build; run the --index-url line again.

Tip: Before the long download, run nvidia-smi (Linux, Windows) and write down the driver version. If PyTorch later says CUDA is not available, that line is the first thing a helper will ask for.

Step 3: Cut a person out of a picture#

Write a short script that loads the 2B model, asks for "person" with the matting task, and saves the result files. The code is the README example with the 2B model, and the repository already ships a sample picture.

Linux#

cat > cutout.py << 'EOF'
from PIL import Image
from wevisone_vl import WeVisOneVL
from wevisone_vl.visualization import save_result

model = WeVisOneVL("tencent/WeVisOne-VL-2B", backend="transformers")
image = Image.open("demo/examples/images/seabreeze.jpg").convert("RGB")
result = model.predict(image, task="matting", query="person")
print(result.response, len(result.masks))
save_result("outputs/matting", image, result)
EOF
python cutout.py

Windows (PowerShell)#

@'
from PIL import Image
from wevisone_vl import WeVisOneVL
from wevisone_vl.visualization import save_result

model = WeVisOneVL("tencent/WeVisOne-VL-2B", backend="transformers")
image = Image.open("demo/examples/images/seabreeze.jpg").convert("RGB")
result = model.predict(image, task="matting", query="person")
print(result.response, len(result.masks))
save_result("outputs/matting", image, result)
'@ | Set-Content -Encoding utf8 cutout.py
python cutout.py

Mac (Terminal)#

Same commands as Linux:

cat > cutout.py << 'EOF'
from PIL import Image
from wevisone_vl import WeVisOneVL
from wevisone_vl.visualization import save_result

model = WeVisOneVL("tencent/WeVisOne-VL-2B", backend="transformers")
image = Image.open("demo/examples/images/seabreeze.jpg").convert("RGB")
result = model.predict(image, task="matting", query="person")
print(result.response, len(result.masks))
save_result("outputs/matting", image, result)
EOF
python cutout.py

The first run downloads about 4.9 GB, so it can take a while. On a Mac or any machine without a GPU, the code switches to 32-bit numbers on the CPU; it should work but expect it to be slow.

Check it worked:

ls outputs/matting

On Windows use Get-ChildItem outputs\matting. Expected files, from the project's save_result code: alpha.png, cutout.png, overlay.png, response.txt, result.json and a masks folder. Open cutout.png in an image viewer: the person should stay and the background should be transparent. The printed response should contain <ref>person</ref> and <box> with numbers between 0 and 1000; the exact numbers are yours, not ours. The number after it counts the masks, so 1 or more is right for a picture with a person.

Tip: The query is a plain phrase. "person" works as a category; "the man on the left" works as a referring expression. If you ask for something that is not there, a correct answer is None and zero masks, not a wrong cutout.

Step 4: Try the other tasks and the command line#

The same call does detection, counting, pointing, text reading and segmentation by changing task and query. The README lists these task values: detection, referring, point, count_box, count_point, ocr, ocr_grounding, segmentation and matting.

The built-in command line tool does one picture per call. Note --model: without it the tool loads the 4B model (the default in cli.py).

Linux#

wevisone-vl --model tencent/WeVisOne-VL-2B --image demo/examples/images/livingroom.jpg \
  --task detection --query "sofa" --output-dir outputs/detection

Windows (PowerShell)#

wevisone-vl --model tencent/WeVisOne-VL-2B --image demo/examples/images/livingroom.jpg `
  --task detection --query "sofa" --output-dir outputs/detection

Mac (Terminal)#

Same commands as Linux:

wevisone-vl --model tencent/WeVisOne-VL-2B --image demo/examples/images/livingroom.jpg \
  --task detection --query "sofa" --output-dir outputs/detection

Check it worked: the folder outputs/detection holds overlay.png with a box drawn around the sofa and a result.json with the prompt, the response and the latency. A bad image path stops the tool at once with --image is not an existing file.

Step 5: Start the local web demo (optional)#

The repository ships a Gradio web page for trying the model in a browser; start it bound to your own machine only. The program's default is --server-name 0.0.0.0 (the README's extra-options example sets it too), which opens the page to everyone on your network. We change it to 127.0.0.1.

Linux#

GRADIO_ANALYTICS_ENABLED=False CUDA_VISIBLE_DEVICES=0 python demo/app.py \
  --model tencent/WeVisOne-VL-2B --server-name 127.0.0.1 --server-port 8080

Windows (PowerShell)#

$env:GRADIO_ANALYTICS_ENABLED = 'False'
$env:CUDA_VISIBLE_DEVICES = '0'
python demo/app.py --model tencent/WeVisOne-VL-2B --server-name 127.0.0.1 --server-port 8080

Mac (Terminal)#

No NVIDIA card, so leave out CUDA_VISIBLE_DEVICES and ask for the CPU:

GRADIO_ANALYTICS_ENABLED=False python demo/app.py \
  --model tencent/WeVisOne-VL-2B --device cpu --server-name 127.0.0.1 --server-port 8080

Check it worked: open http://127.0.0.1:8080 in your browser and upload a picture. To look at the page without loading the model, add --mock; it returns a fake matte and is only a layout preview. Never add --share for private pictures (see Security).

Put it behind HTTPS (only if you share the demo)#

Skip this section if you use the demo on your own computer. The project has no official Docker image and no Traefik or Caddy instructions, so nothing below is official. Keep the demo on 127.0.0.1 and let a reverse proxy face the network.

With Traefik#

Traefik can send traffic to a program that is not in Docker, but we will not invent labels for it. If you already run Traefik, point a router with TLS and a login at http://127.0.0.1:8080. Do not publish port 8080 itself.

Without Traefik: Caddy#

Caddy gets and renews a Let's Encrypt certificate by itself for a public domain. For a home network use tls internal, which is Caddy's own local certificate; trust its root certificate on each device (caddy trust on the Caddy machine).

cutout.example.com {
    reverse_proxy 127.0.0.1:8080
}

Home-only variant:

cutout.home.arpa {
    tls internal
    reverse_proxy 127.0.0.1:8080
}

Run it with caddy run --config Caddyfile (Linux: install the Caddy package first; Windows: winget install CaddyServer.Caddy; Mac: brew install caddy). Check with curl -I https://cutout.example.com/. The demo has no login of its own, so put a login in front or keep it private.

Security and testing#

The risks are the web demo's open address, the code the model downloads, and the public share link.

• Bind address. The demo's default is --server-name 0.0.0.0 (in demo/app.py), although Gradio itself defaults to 127.0.0.1. Always set 127.0.0.1.
• --share. Gradio says share links are publicly accessible, "meaning that anyone can use your model", and they expire after one week. They go through Gradio's servers as a proxy. Do not use this for private pictures.
• trust_remote_code=True. engine.py loads the model with this flag, which lets code from the model repository run on your machine. Pin the model revision (see Updates) and read what you pin if that matters to you.
• Analytics. Gradio collects usage statistics by default; set GRADIO_ANALYTICS_ENABLED=False. Hugging Face libraries also have telemetry; HF_HUB_DISABLE_TELEMETRY=1 turns it off.
• Ports and files. Keep 8080 off the internet. The outputs folder holds your pictures in plain files.

What leaves your machine:

We did not audit the code or watch the network. The "nothing leaves" line comes from how the code is built: it loads files and runs locally.

Offline mode after the first download:

HF_HUB_OFFLINE=1 python cutout.py

On Windows PowerShell set $env:HF_HUB_OFFLINE = '1' first. Hugging Face's own words: "no HTTP calls will be made to the Hugging Face Hub... only the cached files will be accessed". If the cache is empty, you get an error instead.

Quick tests: the Step 3 outputs/matting files exist, and ss -ltn | grep 8080 (Linux/Mac) or Get-NetTCPConnection -LocalPort 8080 (Windows) shows 127.0.0.1 only while the demo runs.

What the numbers mean#

The authors report lower matting error than SAM 2-based matting, but these are their own tests on box and text prompts, not independent ones. In the table, MAD is the average difference to the true matte (times 1000), so lower is better. Source: the repository README and the paper's Table 8, read October 2026.

Read these carefully:

• SAM2-Matting means SAM first makes a mask from your box, then a matting step runs. The README writes "SAM2-Matting (SAM3)" for the SAM 3 variant, which is easy to mistake for SAM 2. The paper tests SAM 2.1 T, SAM 2.1 B+ and SAM 3.
• WeRefMat is the authors' own benchmark. The three older sets are public.
• This is matting, not plain segmentation. The project's segmentation tables compare with other models (LISA, GSVA, UniPixel) and not with SAM 2, so there is no official "2B beats SAM 2 at segmentation" number.
• The big README table is for the 4B model. For the 2B, see the full tables: for example detection F1 on COCO 55.2 and on LVIS 53.3, referring-segmentation cIoU 74.9 on RefCOCOg test.
• No independent test exists yet as far as we found. The project is from October 2026 (about a week old).

Can you use it commercially?#

The licence text allows it, with no extra restriction from Tencent, but we are not lawyers. The Hub's LICENSE file for the 2B says:

"WeVisOne-VL-2B is licensed under Apache-2.0, except for the third-party components listed below, which remain licensed under their respective original terms. WeVisOne-VL-2B does not impose any additional restrictions beyond those specified in the original licenses of these third-party components."

The only listed third-party part is Qwen3-VL-2B-Instruct (Alibaba Cloud), also under Apache-2.0. Apache-2.0 section 2 grants "a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work". Keep the licence notices, mark files you changed, and do not use Tencent's names as your own. Two things the files do not say: where the roughly 100 million training images came from and under which licences. The model card also states the release "is not intended to make autonomous high-impact decisions or to identify individuals".

Common mistakes#

Tip: Start with the sample picture and the "person" query from Step 3. If that works, then try your own files; if it fails, you know it is the setup and not your picture.

Tip: Big pictures cost memory. The loader accepts max_pixels (default 2,073,600 pixels in engine.py); lower it, for example WeVisOneVL("tencent/WeVisOne-VL-2B", max_pixels=1024*32*32), if you run out of memory.

Troubleshooting#

Known bugs#

As of October 2026 the tracker has 6 open issues and none is a bug report. They are requests: ONNX inference (#5), fine-tuning code (#7, still unchecked in the README's TODO list), a ModelScope copy (#6), Papers with Code results (#1), LensBench results (#4) and a Japanese README (#8). Issue #2 confirms visual prompting was trained but not benchmarked.

Diagnostics:

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
nvidia-smi
pip list | grep -i -E "torch|transformers|gradio"

On Windows use pip list | Select-String "torch|transformers|gradio".

Updates and backups#

There are no releases to follow, so pin the model and the code yourself and update on purpose. In October 2026 the 2B model's Hub revision was fcf8c8ca7fa481b514005ef591c8b72bed39618a and the code's latest commit was 9a6bcd8. The loader accepts revision=, so a script can keep that exact model:

model = WeVisOneVL("tencent/WeVisOne-VL-2B", revision="fcf8c8ca7fa481b514005ef591c8b72bed39618a")

Update the code with git pull, then pip install -v -e . again. Read the commit list first; the project is new and changing.

Back up your scripts and results, not the weights (they can be downloaded again):

Linux#

mkdir -p backups
tar -czf "backups/wevisone-$(date +%F).tar.gz" cutout.py outputs

Windows (PowerShell)#

New-Item -ItemType Directory -Force backups | Out-Null
Compress-Archive -Path cutout.py, outputs -DestinationPath "backups\wevisone-$(Get-Date -Format yyyy-MM-dd).zip"

Mac (Terminal)#

Same commands as Linux:

mkdir -p backups
tar -czf "backups/wevisone-$(date +%F).tar.gz" cutout.py outputs

Schedule it with cron at 03:00 (0 3 * * *) on Linux and Mac, or Task Scheduler at 03:00 on Windows. Restore test: extract the archive into an empty folder and open one cutout.png.

Performance and resources#

No official speed or memory figures exist, and we measured none. What is documented: 2,441,744,384 parameters in BF16, a 4.9 GB weights file, and the paper's own tests were run with greedy decoding and at most 1,960 visual tokens per image. The 4B model is 9.7 GB.

Knobs from the code: max_pixels and min_pixels set the picture size the model sees; decoder_batch_size (default 16) sets how many masks the pixel decoder handles at once; max_new_tokens defaults to 4096 in the command line tool; the vllm backend exists for Linux with a GPU. Watch memory with nvidia-smi -l 2 on Linux and Windows, top on Linux and Activity Monitor on a Mac.

FAQ#

Can it replace SAM 2? For text-driven cutouts and mattes, the authors' tests say it is stronger than SAM 2-based matting. It has no video mode, and it is a new project with no independent tests yet. Treat it as one more tool, not a drop-in swap.

Does it work without a GPU? The code falls back to the CPU, so it should run, slowly and with more memory. The official docs name no minimum, so we cannot promise it.

Does it send my pictures anywhere? Not by design. Only the first download contacts Hugging Face. The --share option and the hosted Hugging Face Space do send your pictures out.

Which model should I pick, 2B or 4B? The 2B is half the download. The 4B is better on most of the authors' tables; for matting the 2B is ahead on two of the three box-prompt tests. Try 2B first.

Is there a Docker image or an Ollama model? Not in the official repository. Anything claiming to be one did not come from Tencent's documentation.

Sources#

• WeVisOne-VL repository: README, LICENSE, requirements.txt, pyproject.toml, source code (read October 2026; issues #2, #3, #6 and the open-issue list)
• WeVisOne-VL-2B model card and LICENSE on Hugging Face (read October 2026)
• WeVisOne-VL paper, arXiv 2610.31329 and project page (read October 2026)
• PyTorch install selector and the CUDA 12.8 wheel index (read October 2026)
• Gradio: sharing your app and environment variables (read October 2026)
• Hugging Face Hub environment variables (read October 2026)
• Caddy docs: automatic HTTPS and reverse_proxy (read October 2026)
• Checked against the official docs on 2026-10-30. Not run by DIY Lab Hub: no GPU machine was used for this page, so every command is confirmed against the project's own files and no output is ours.