Classification Pro

imgproxy can classify images by assigning them to predefined categories based on the overall content of the image. Unlike object detection, which answers “Where is a cat in this image?”, image classification answers “Is there a cat in this image?” by labeling the entire image as a whole.

Specialized image classification models are typically faster and simpler than object detection because they do not need to locate objects within the image. They can also classify images based on broader concepts such as scenes, activities, or safety categories (e.g., NSFW vs. SFW), and require only labeled images for training instead of annotated bounding boxes.

Configuration

tip

If you're using an imgproxy Pro Docker image with a tag suffixed with -ml, a basic classification model is included. For advanced classification, you may want to configure your own model.

The list of classes available in the bundled model

Accordion
Adhesive tape
Aircraft
Alarm clock
Alpaca
Ambulance
Ant
Antelope
Apple
Armadillo
Artichoke
Axe
Backpack
Bagel
Baked goods
Balance beam
Ball (Object)
Balloon
Banana
Band-aid
Banjo
Barge
Barrel
Baseball bat
Baseball glove
Bat (Animal)
Bathroom accessory
Bathroom cabinet
Bathtub
Beaker
Bear
Bed
Bee
Beehive
Beer
Beetle
Bell pepper
Belt
Bench
Bicycle
Bicycle helmet
Bidet
Billboard
Billiard table
Binoculars
Bird
Blender
Blue jay
Boat
Bomb
Book
Bookcase
Boot
Bottle
Bottle opener
Bow and arrow
Bowl
Bowling equipment
Box
Boy
Brassiere
Bread
Briefcase
Broccoli
Bronze sculpture
Brown bear
Bull
Burrito
Bus
Bust
Butterfly
Cabbage
Cabinetry
Cake
Cake stand
Calculator
Camel
Camera
Can opener
Canary
Candle
Candy
Cannon
Canoe
Cantaloupe
Car
Carrot
Cart
Cassette deck
Castle
Cat
Cat furniture
Caterpillar
Cattle
Ceiling fan
Cello
Centipede
Chainsaw
Chair
Cheese
Cheetah
Chest of drawers
Chicken
Chime
Chisel
Chopsticks
Christmas tree
Clock
Closet
Coat
Cocktail
Cocktail shaker
Coconut
Coffee (drink)
Coffee cup
Coffee table
Coffeemaker
Coin
Common fig
Common sunflower
Computer keyboard
Computer monitor
Computer mouse
Container
Convenience store
Cookie
Cooking spray
Corded phone
Cosmetics
Couch
Countertop
Cowboy hat
Crab
Cream
Cricket ball
Crocodile
Croissant
Crown
Crutch
Cucumber
Cupboard
Curtain
Cutting board
Dagger
Dairy Product
Deer
Desk
Dessert
Diaper
Dice
Digital clock
Dinosaur
Dishwasher
Dog
Dog bed
Doll
Dolphin
Door
Door handle
Doughnut
Dragonfly
Drawer
Dress
Drill (Tool)
Drink
Drinking straw
Drum
Duck
Dumbbell
Eagle
Earring
Egg
Elephant
Envelope
Eraser
Face powder
Facial tissue holder
Falcon
Fast food
Fax
Fedora
Filing cabinet
Fire hydrant
Fireplace
Fish
Fixed-wing aircraft
Flag
Flashlight
Flower
Flowerpot
Flute
Flying disc
Food processor
Football
Football helmet
Footwear
Fork
Fountain
Fox
French fries
French horn
Frog
Fruit
Frying pan
Garden Asparagus
Gas stove
Giraffe
Girl
Glasses
Glove
Goat
Goggles
Goldfish
Golf ball
Golf cart
Gondola
Goose
Grape
Grapefruit
Grinder
Guacamole
Guitar
Hair dryer
Hair spray
Hamburger
Hammer
Hamster
Hand dryer
Handbag
Handgun
Harbor seal
Harmonica
Harp
Harpsichord
Hat
Headphones
Heater
Hedgehog
Helicopter
Helmet
High heels
Hiking equipment
Hippopotamus
Honeycomb
Horizontal bar
Horse
Hot dog
House
Houseplant
Humidifier
Ice cream
Indoor rower
Infant bed
Insect
Ipod
Isopod
Jacket
Jacuzzi
Jaguar (Animal)
Jeans
Jellyfish
Jet ski
Jug
Juice
Kangaroo
Kettle
Kitchen & dining room table
Kitchen appliance
Kitchen knife
Kitchen utensil
Kite
Knife
Koala
Ladder
Ladle
Ladybug
Lamp
Lantern
Laptop
Lavender (Plant)
Lemon (plant)
Leopard
Light bulb
Light switch
Lighthouse
Lily
Limousine
Lion
Lipstick
Lizard
Lobster
Loveseat
Luggage and bags
Lynx
Magpie
Man
Mango
Maple
Maraca
Measuring cup
Mechanical fan
Medical equipment
Microphone
Microwave oven
Milk
Miniskirt
Mirror
Missile
Mixer
Mixing bowl
Mobile phone
Monkey
Moths and butterflies
Motorcycle
Mouse
Muffin
Mug
Mule
Mushroom
Musical instrument
Musical keyboard
Nail (Construction)
Necklace
Nightstand
Oboe
Office building
Orange (fruit)
Organ (Musical Instrument)
Ostrich
Otter
Oven
Owl
Oyster
Paddle
Palm tree
Pancake
Panda
Paper cutter
Paper towel
Parachute
Parking meter
Parrot
Pasta
Pastry
Peach
Pear
Pen
Pencil case
Pencil sharpener
Penguin
Perfume
Person
Personal flotation device
Piano
Picnic basket
Picture frame
Pig
Pillow
Pineapple
Pitcher (Container)
Pizza
Pizza cutter
Plastic bag
Plate
Platter
Polar bear
Pomegranate
Popcorn
Porch
Porcupine
Poster
Potato
Power plugs and sockets
Pressure cooker
Pretzel
Printer
Pumpkin
Punching bag
Rabbit
Raccoon
Racket
Radish
Ratchet (Device)
Raven
Rays and skates
Red panda
Refrigerator
Remote control
Reptile
Rhinoceros
Rifle
Ring binder
Rocket
Roller skates
Rose
Rugby ball
Ruler
Salad
Salt and pepper shakers
Sandal
Sandwich
Saucer
Saxophone
Scale
Scarf
Scissors
Scoreboard
Scorpion
Screwdriver
Sculpture
Sea lion
Sea turtle
Seafood
Seahorse
Segway
Serving tray
Sewing machine
Shark
Sheep
Shelf
Shellfish
Shirt
Shorts
Shotgun
Shower
Shrimp
Sink
Skateboard
Ski
Skirt
Skull
Skunk
Skyscraper
Slow cooker
Snack
Snail
Snake
Snowboard
Snowman
Snowmobile
Snowplow
Soap dispenser
Sock
Sofa bed
Sombrero
Sparrow
Spatula
Spice rack
Spider
Spoon
Sports uniform
Squash (Plant)
Squid
Squirrel
Stairs
Stapler
Starfish
Stationary bicycle
Stethoscope
Stool
Stop sign
Strawberry
Street light
Stretcher
Studio couch
Submarine
Submarine sandwich
Suit
Suitcase
Sun hat
Sunglasses
Surfboard
Sushi
Swan
Swim cap
Swimming pool
Swimwear
Sword
Syringe
Table
Table tennis racket
Tablet computer
Tableware
Taco
Tank
Tap
Tart
Taxi
Tea
Teapot
Teddy bear
Telephone
Television
Tennis ball
Tennis racket
Tent
Tiara
Tick
Tie
Tiger
Tin can
Toaster
Toilet
Toilet paper
Tomato
Toothbrush
Torch
Tortoise
Towel
Tower
Toy
Traffic light
Traffic sign
Train
Training bench
Treadmill
Tree
Tree house
Tripod
Trombone
Trousers
Truck
Trumpet
Turkey
Turtle
Umbrella
Unicycle
Van
Vase
Vegetable
Violin
Volleyball (Ball)
Waffle
Waffle iron
Wall clock
Wardrobe
Washing machine
Waste container
Watch
Watermelon
Weapon
Whale
Wheelchair
Whisk
Whiteboard
Willow
Window
Window blind
Wine
Wine glass
Wine rack
Winter melon
Wok
Woman
Wood-burning stove
Woodpecker
Worm
Wrench
Zebra
Zucchini

You need to define the following config variables to enable object classification:

• IMGPROXY_CLASSIFICATION_NET: a path to the classification neural network model in ONNX format

• IMGPROXY_CLASSIFICATION_CLASSES: a path to the class names file

• IMGPROXY_CLASSIFICATION_NET_SIZE: the size of the neural network input. The width and the heights of the inputs should be the same, so this config value should be a single number. Default: 224

• IMGPROXY_CLASSIFICATION_THRESHOLD: classifications with confidence below this value will be discarded. Default: 0.5

• IMGPROXY_CLASSIFICATION_NORMALIZATION: the normalization type to apply to the input image. Possible values:

  • none: no normalization
  • half: normalize to [0, 1] range
  • full: normalize to [-1, 1] range
  • imagenet: normalize using ImageNet mean and standard deviation

  Default: none

• IMGPROXY_CLASSIFICATION_LAYOUT: the data layout of the neural network input. Possible values:

  • nhwc: channels last (TensorFlow default)
  • nchw: channels first (PyTorch default)

  Default: nhwc

Class names file

The class names file maps the neural network's class indexes to human-readable class names. The path to the class names file should be defined in the IMGPROXY_CLASSIFICATION_CLASSES config variable.

The class names file should contain one class name per line. The class names should match the order of the classes in the neural network output. Example:

person
bicycle
car

Getting classification info

Object classification is available via the info handler using the cl (classify) endpoint. Fetch classification results by specifying the number of top classes to return:

/info/.../cl:5/...

Where 5 is the number of top classes to return.

The response is an array of objects, each containing:

• class_id: The numeric ID of the class
• name: The class name from the class names file
• confidence: The confidence score for this classification

See the getting info documentation for more details.