assignment 11.2 image labeling

Data Topics

• Data Architecture
• Data Literacy
• Data Science
• Data Strategy
• Data Modeling
• Governance & Quality
• Data Education
• Smart Data News, Articles, & Education

Complete Guide to Image Labeling for Machine Learning

Image labeling enables you to tag and identify specific details in an image. In computer vision, image labeling involves adding specific tags to raw data, including videos and images. Each tag represents a certain object class associated with this data.  Supervised machine learning (ML) models utilize labels to learn to identify a certain object class within unclassified […]

Image labeling enables you to tag and identify specific details in an image. In computer vision, image labeling involves adding specific tags to raw data, including videos and images. Each tag represents a certain object class associated with this data. 

Image annotation is a  type of image labeling  used to create datasets for computer vision models. You can split these datasets into training sets to train ML models and test or validate datasets before using them to evaluate model performance. 

Data scientists and  machine learning engineers  employ these datasets to train and evaluate ML models. At the end of the training period, the model can automatically assign labels to unlabeled data.

Why Is Image Labeling Important for AI and Machine Learning?

Image labeling enables supervised machine learning models to achieve computer vision capabilities. Data scientists use image labeling to train ML models to:

• Label an entire image to learn its meaning
• Identify object classes within an image

Essentially, image labeling enables ML models to understand the content of images. Image labeling techniques and tools help ML models capture or highlight specific objects within each image, making images readable by machines. This capability is crucial for developing functional AI models and improving computer vision. 

Image labeling and annotation enable object recognition in machines to improve computer vision accuracy. Using labels to train AI and ML helps the models learn to detect patterns. The models run through this process until they can recognize objects independently.

Types of Computer Vision Image Labeling

Image classification.

You can annotate data for image classification by adding a tag to an image. The number of unique tags in a database matches the number of classes the model can classify.

Here are the three key classification types:

• Binary class classification:  Includes only two tags
• Multiclass classification:  Includes multiple tags
• Multi-label classification:  Each image can have more than one tag

Image Segmentation

Image segmentation involves using computer vision models to separate objects in an image from their backgrounds and other objects. 

It usually requires creating a pixel map the same size as the image, using the number 1 to indicate the object is present and the number 0 to indicate no annotations exist. 

Segmenting multiple objects in the same image involves concatenating pixel maps for each object channel-wise and using the maps as ground truth for the model.

Object Detection

Object detection involves using computer vision to identify objects and their specific locations. Unlike image classification, object detection processes annotate each object using bounding boxes. 

A bounding box consists of the smallest rectangular segment containing an object in the image. Bounding box annotations are often accompanied by tags, providing each bounding box with a label in the image.

The coordinates of bounding boxes and associated tags are usually stored in a separate JSON file in a dictionary format. Typically, the image number or image ID is the dictionary’s key.

Pose Estimation

Pose estimation involves using computer vision models to estimate a person’s pose in an image. It works to detect key points in the human body and correlate them to estimate the pose, meaning the key points serve as the corresponding ground truth for pose estimation. 

Pose estimation requires labeling simple coordinate data with tags. Each coordinate indicates the location of a certain key point, which is identified by a tag in the image.

Effective Image Labeling for Computer Vision Projects

The following best practices can help you perform more effective image selection and labeling for computer vision models:

• Include both machine learning and domain experts in initial image selection.
• Start with a small batch of images, annotate them, and get feedback from all stakeholder to prevent misunderstandings and understand exactly what images are needed.
• Consider what your model needs to detect, and ensure you have sufficient variation of appearance, lighting, and image capture angles.
• When detecting objects, ensure you select images of all common variations of the object – for example, if you are detecting cars, ensure you have images of different colors, manufacturers, angles, and lighting conditions.
• Go through the dataset at the beginning of the project, consider cases that are more difficult to classify, and come up with consistent strategies to deal with them. Ensure you document and communicate your decisions clearly to the entire team.
• Consider factors that will make it more difficult for your model to detect an object, such as occlusion or poor visibility. Decide whether to exclude these images, or purposely include them to ensure your model can train on real-world conditions.
• Pay attention to quality, perform rigorous QA, and prefer to have more than one data labeler work on each image, so they can verify each other’s work. Mismatched labels can negatively affect  data quality  and will hurt the model’s performance.
• As a general rule, exclude images that are not sharp enough, or do not have enough visual information. But take into account that the model will not be able to work with these types of images in real life. 
• Use existing datasets – these typically contain millions of images and dozens or hundreds of different categories. Two common examples are ImageNet and COCO. 
• Use transfer learning techniques to leverage visual knowledge from similar, pre-trained models and use it for your own models.

In this article, I explained why image labeling is critical for machine learning models related to computer vision. I discussed the important types of image labeling – image classification, image segmentation, object detection, and pose estimation. 

Finally, I provided some best practices that can help you make image labeling more effective, including:

• Include experts in initial image selection
• Start with a small batch of images
• Ensure you capture all common variations of the object
• Consider edge cases and how to deal with them
• Consider factors like occlusion or poor visibility
• Pay attention to quality
• Use existing datasets if possible and use transfer learning to leverage knowledge from similar, pre-trained models for your own models.

I hope this will be useful as you advance your use of image labeling for machine learning.

Leave a Reply Cancel reply

You must be logged in to post a comment.

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

• Biochemistry

Labelling instructions matter in biomedical image analysis

• Nature Machine Intelligence 5(3):1-11

• German Cancer Research Center

• This person is not on ResearchGate, or hasn't claimed this research yet.

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Vivienn Weru

• Rohan Shawn Sunil
• Hilbert Yuen In Lam
• Marek Mutwil

• Gabriel Lepetit-Aimon

• Marie Carole Boucher

• Gauhar Dunenova
• Zhanna Kalmataeva

• Alaa Youssef
• David Fronk
• John Nicholas Grimes
• David B. Larson
• Malte Tölle
• Philipp Garthe

• Fernando Navarro
• Sebastian Eble
• Isabella Galter
• Elida Schneltzer
• Carsten Marr
• Martin Hrabě de Angelis

• Amandalynne Paullada
• Inioluwa Deborah Raji
• Emily M. Bender

• Alexander J. Quinn
• Beverly Freeman

• Sonia Phene
• Rory Sayres
• Bjoern H. Menze
• Ian Kivlichan
• Rachel Rosen
• Emily Denton
• MED IMAGE ANAL

• John P. Cunningham

• William Hiesinger
• ARTIF INTELL MED

• Howard Morgan
• Michael Dohopoloski
• Steve Jiang
• Samuel Budd

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

How to Label Image Data for Computer Vision Models

Creating a high quality dataset for computer vision is essential to having strong model performance .

In addition to collecting images that are as similar to your deployed conditions as possible, labeling images carefully and accurately is essential.

But how do you label images? What should you keep in mind to label images well? We're going to answer those questions in this post. By the end of this post, you'll have a few tips to keep in mind so that your image annotations are correct and helpful a model.

Check out the video version of this article on our YouTube channel .

What is image labeling?

Image labelling is when you annotate specific objects or features in an image. Image labels teach computer vision models how to identify a particular object in an image. For example, in a set of aerial view images, you might annotate all of the trees. The labels will help a model understand what a tree is.

Image labelling can be done with a variety of annotation tools. These annotation tools let you draw specific boundaries around objects. These boundaries are called "bounding boxes". Each bounding box is given a label so that the model can differentiate different objects. For example, all trees might be labeled as "tree" and all houses might be annotated as "house".

The quality of your image labels will directly affect the accuracy of a trained model. Using the right labeling and annotation strategy, you can produce a high-quality dataset that will help a model better learn how to identify the objects you have labelled.

Caveat: Labeling instructions depend on your task

While the below best practices are generally true, it is important to note that labeling instructions are highly dependent on the nature of the task at hand.

Moreover, images labeled for one task, may not be suitable for another task – re-labeling is not uncommon. It is best to think of a dataset and its labels as something alive: constantly changing and improving to fit the task at hand.

Label and Annotate Data with Roboflow for free

Use Roboflow to manage datasets, label data, and convert to 26+ formats for using different models. Roboflow is free up to 10,000 images, cloud-based, and easy for teams.

How to label images for computer vision tasks

With that in mind, lets walkthrough a few tips on how to effectively label images.

Unsure which images to label first? Consider how to use active learning in computer vision .

1. Label Every Object of Interest in Every Image

Computer vision models are built to learn what patterns of pixels correspond to an object of interest.

Because of this, if we're training a model to identify an object, we need to label every appearance of that object in our images. If we do not label the object in some images, we will be introducing false negatives to our model.

For example, in a chess piece dataset , we need to label the appearance of every single piece on the board – we would not label only some of, say, the white pawns.

2. Label the Entirety of an Object

Our bounding boxes should enclose the entirety of an object of interest. Labeling only a portion of the object confuses our model for what a full object constitutes.

In our chess dataset, for example, notice how each piece is fully enclosed in a bounding box.

3. Label Occluded Objects

Occlusion is when an object is partially out of view in an image due to something blocking it in a photo. It is best to label even occluded objects.

Moreover, it is commonly best practice to label the occluded object as if it were fully visible – rather than drawing a bounding box for only the partially visible portion of the object.

For example, in the chess dataset, one piece will regularly occlude the view of another. Both objects should be labeled, even if the boxes overlap. (It is a common misconception that boxes cannot overlap.)

4. Create Tight Bounding Boxes

Bounding boxes should be tight around the objects of interest. (But, you should never have a box so tight that it is cutting off a portion of the object.) Tight bounding boxes are critical to helping our model learn, precisely, which pixels constitute an object of interest vs irrelevant portions of an image.

5. Create Specific Label Names

When determining a given object's label name, it is better to error on the side of being more specific rather than less. It is always easier to remap label classes to be more general, whereas being more specific requires relabeling.

For example, imagine you are building a dog detector . While every object of interest is a dog, it may be wise to create a class for labrador and poodle . In initial model building, our labels could be combined to be dog . But, if we had started with dog and later realized having individual breeds is important, we would have to relabel our dataset altogether.

In our chess dataset, for example, we have white-pawn and black-pawn . We could always combine these to be pawn , or even combine all classes to be piece .

6. Maintain Clear Labeling Instructions

Inevitably, we will need to add more data to our dataset – this is a key element to model improvement. Tactics like active learning ensure that we spend our time labeling intelligently. As such, having clear, shareable, and repeatable labeling instructions is essential for both our future selves and coworkers to create and maintain high quality datasets.

Many of the techniques we've discussed here should be included: label the entirety of an object, make labels tight, label all objects, etc. It is always better to err on the side of more specificity than less.

7. Label Faster with Roboflow’s Professional Labelers

Through Roboflow’s Outsource Labeling service, you can work directly with professional labelers to annotate projects of all sizes. Roboflow manages workforces of experts who are trained in using Roboflow’s platform to curate datasets faster and cheaper. 

The first step in getting started with Outsource Labeling is to fill out the intake form with your project’s details and requirements. From there, you will be connected with a team of labelers to directly work with on your labeling project(s).

When working with professional labelers, clearly documenting your instructions is an essential part of the process. We often see that the most successful labeling projects are the ones in which well documented instructions are provided upfront, a period of initial feedback takes place with the labelers regarding an initial batch of images, and then the labeling volume is significantly ramped up. Read our guide to writing labeling instructions for more information about how to write informative instructions..

As part of the Outsource Labeling service, you will also be working with a member of the Roboflow team to help guide your labeling strategy and project management to ensure you are curating the highest quality dataset possible.

8. Get the Most Out of Roboflow’s Annotation Tools

Roboflow has a suite of annotation tools to make the process of labeling data more efficient and accurate.

• Label Assist : a tool that uses model checkpoints (i.e. a previous version of your model) to recommend annotations.
• Smart Polygon : a tool that leverages the Segment Anything Model to create polygon annotations with a few clicks.
• Auto Label (Beta): a tool that leverages large foundational models to label images based on prompted descriptions of each Class.
• Commenting : a feature that allows for seamless cross-team collaboration throughout the labeling process. 

When we're ready to scale up our labeling efforts either with teammates or an outsourced workforce, we manage large scale labeling projects with team workflows and label-only users .

As always, happy building!

Cite this Post

Use the following entry to cite this post in your research:

Joseph Nelson . (Jan 5, 2024). How to Label Image Data for Computer Vision Models. Roboflow Blog: https://blog.roboflow.com/tips-for-how-to-label-images/

Discuss this Post

If you have any questions about this blog post, start a discussion on the Roboflow Forum .

• Dataset Management
• Roboflow Annotate

More About Dataset Management

What is active learning the ultimate guide., how to make multiplayer rock, paper, scissors with ai, how to create a workout pose correction tool, how to import hugging face datasets to roboflow, what is the open images dataset a deep dive., how to train rt-detr on a custom dataset with transformers.

Impact on AI

Get comprehensive guide for superior RLHF. Train safer, more accurate models with expert data.

Get the guide

Natalie Kudan

Dec 27, 2022

Essential ML Guide

How to label images for machine learning

For those of you looking to learn more about how to perform image labeling to train a machine learning model, this article is for you. We’ve outlined what labeling images is, what it’s used for, what the typical process looks like, what the types and methods are, and how to optimize with crowdsourcing . Keep reading to find out more. 

What is image labeling?

Image labeling is a type of data labeling where the goal is to add meaningful information to images. The task in general is to identify certain features or objects in an image, and then to use this information to select said objects in an image, to classify the image according to the presence of these features. This type of labeling is often used to obtain training data for machine learning models, especially in the field of computer vision.

In other words, it’s when you annotate certain objects or features in an image. These labels teach a computer vision model how to identify a particular object. For example, in a series of high-angle images of a city, you could annotate all the skyscrapers. These labels help a model determine what a skyscraper is.

Image annotation is used to create datasets with different objects for computer vision models, which are split into training sets — for initial model training — and test/validation sets — to evaluate model performance. Data scientists use the dataset to train and evaluate their model. Then the model can automatically assign labels to unlabeled data.

By using the right image labeling strategy, you can create a high-quality dataset that will help a model better learn how to identify objects. Labeling images is a dynamic process which machine learning engineers are continuously adapting and improving upon.

How to label image data for machine learning

To label images for training a computer vision model, you need to follow these steps.

1. Define which kind of data you need for model training

The type of data labeling task you will do will depend on that. For example, in some cases you might need sets of images representing certain categories (image classification task), in other cases you might need images with certain types of objects identified and selected (object detection task).

2. Define the characteristics of labeled data your model needs

For an image classification task, you need to define classes. For object detection tasks, the rules of markup: do you need precise selection via polygons, or is it enough to use bounding boxes?

3. Decide how much labeled data of each type you need

Before collecting and labeling data, you need to understand how much of each type of data you need to train a balanced and unbiased ML model. You wouldn't want to skew your model's performance due to imbalanced training data.

4. Choose the optimal way to label training data

In general, there are a few ways: human data labeling or automation. Human labeling is more time consuming and expensive, but tends to be more reliable. If you decide you need human input, there is in-house labeling, outsourcing, and crowdsourcing. For more information on how AI-powered businesses label data today, check out our blog post .

5. Decompose the labeling task

If you decide to employ human data labeling to ensure high-quality results, you'll need to break your image labeling task down into steps that are clear enough for anyone to handle. To ensure optimal labeling, break your task down into parts by replacing one large problem with a series of smaller, separate problems that are easier to solve.

6. Write clear instructions

The more straightforward and clear your labeling instructions are, the more reliable the whole process will be. Oftentimes, things that seem obvious to you might not be clear to everyone else. Write concise and comprehensive instructions , provide examples, and foresee common mistakes.

7. Set up quality control

Think in advance about what you will do to ensure the quality of labeled data , preferably automatically, without the need to check the results yourself. This usually means you need to create a pipeline: a series of labeling and verification steps for your image labeling process. For example, divide your object detection task between three groups of people: the first person defines whether the desired object is present on an image, the second person selects the said object, and the third person checks if the object has been selected correctly.

Image labeling also plays a significant role in AI and ML as a key component of developing supervised models with computer vision capabilities. Image labeling helps train machine learning models to label images or identify classes of objects within an image. Training with labels helps these models identify patterns until they can recognize objects on their own.

What is image labeling and annotation used for?

Image annotation is a dynamic process that involves labeling digital images and is a vital part of training computer vision models that process image data for object detection, classification, segmentation, and more. A dataset of images that have been labeled and annotated to identify and classify an object, for example, is required to train an object detection model. This kind of computer vision projects is an increasingly important technology. For example, manufacturers of self-driving cars rely on millions of correctly labeled data points to ensure the safety and efficiency of their vehicles.

Image labeling is used across a wide variety of industries for various computer vision tasks such as:

Retail and e-commerce

Product recognition on store shelves

Virtual fitting rooms

People counting for retail stores

Transportation

Pedestrian detection

Traffic prediction

Parking occupancy detection

Road condition monitoring

Manufacturing

Personal protective equipment detection

Facial feature detection

Iris recognition

Agriculture

Plant disease detection

Object detection in agriculture

Logo recognition

Methods used in image labeling

Image annotation sets a standard that computer vision algorithms try to learn from. Therefore, accurate labeling is essential in training neural networks. There are three methods for image labeling: manual, semi-automated, and synthetic.

Manual image annotation

This process involves manually defining labels for an entire image or drawing regions in an image and adding textual descriptions for each region. There is a special kind of computer vision annotation tool that allow operators to rotate through multiple images, draw regions (bounding boxes or polygons) on an image and assign labels, and save this data to a standardized format that can be used for data training.

However, an in-house approach to manual image annotation has some drawbacks: labels can be inconsistent when multiple annotators are involved, and it’s time consuming, costly, and difficult to scale for large datasets. To ensure consistency, annotators must be provided with clear instructions and consideration needs to be given to quality control of the labeling.

Semi-automated image annotations

An automated image annotation tool can help manual annotators by attempting to detect object boundaries in an image and providing a starting point for the annotator. The algorithms of image annotation software are not 100% accurate, but they can save time for human annotators by providing at least a partial map of objects in the image.

Synthetic image labeling

As an alternative to manual image annotation, synthetic image labeling is an accurate and cost-effective technique. It involves automatically generating images that are similar to real-life objects or human faces. The main benefit of synthetic images is that labels are known in advance.

How to label images via crowdsourcing

Scaling data labeling from a few in-house labelers to an industrial solution would require large data labeling teams as well as dozens of managers to supervise this vast workforce. Driving quality for in-house data labeling means dramatically increasing the time involved in labeling that data, making the entire image labeling process slow and costly.

However, there are alternative ways to label data for ML models . One of them is crowdsourcing. Crowdsourcing refers to a specific process of labeling data that employs many annotators who have signed up on a particular platform. Simply put, teams working with artificial intelligence post unlabeled data and labeling tasks, and people choose and complete tasks they are interested in.

The main challenge lies in correctly formulating a small, simple task. You need a specialist to correctly configure the data annotation pipeline and quality control. Then you can scale annotation and get large volumes of marked-up data quickly, efficiently, and inexpensively.

Overlap is the key to crowdsourcing and is defined as the number of annotators who should complete each task in a pool. Most commonly, it’s set to three. Toloka assigns confidence to Toloker responses for complex image classification tasks. When confidence drops below a specified level, Toloka increases the overlap value until the confidence reaches the set value or the overlap reaches the predefined maximum.

What does the typical image annotation process look like?

We’ve outlined the standard image annotation process below using an example of image markup via Toloka’s data labeling platform .

Step 1: Decompose the task and classify content

In any image classification task, start with the main question you want to ask. If it’s complex, you may want to break the job down into subtasks. After you’ve defined the question, ask yourself what classes you expect — which can help you define the answers, prepare a task interface, and write instructions for annotators.

Example : If you want to create a dataset with marked-up photos of cars, you can assign three consecutive tasks to three groups of annotators. The first task would be to select all the images showing a car, the second to highlight that required object (or multiple objects) with a polygon, and the third to check that the car is indeed highlighted correctly.

Step 2: Prepare instructions for annotators

The more complete and clear your instructions are for annotators, the better labeling quality you will get. Mix in some control tasks with the real ones. That way, you can compare annotator answers with the answers to the control tasks to get an idea of labeling accuracy.

To improve your instructions, you can run labeling for a small part of your dataset without control tasks first. Read through the results. This will reveal the most common errors and identify cases the instructions don’t cover.

Example : A potential error in the scenario mentioned above is if toy cars are also included in the images. This leads to the question: should toy cars be counted or not?

Step 3: Direct markup

The task interface defines what the job looks like for the annotators, and the logic they follow to process their responses. If it’s simple and clean, they can work faster and on different devices. Adding automatic verifications improves labeling quality.

As another example, read our case study on image classification for self-driving cars.

Types of image labeling

There are several different image labeling approaches, which we’ve outlined below as well as use cases from our Toloka platform.

Image classification

Image classification algorithms receive images as an input and automatically classify them into one of several labels (also known as classes). Machine learning models must learn to recognize such objects in the images themselves. To create a training dataset for image classification, you need to manually review images and annotate them with labels used by the algorithm.

How to do it : Take a look at an image and see what’s being shown – for example, a cat or a dog. Then choose which class the object in the image belongs to.

On Toloka : Match visual content to predefined categories. Use for search relevance , recommendation systems, image moderation, and more.

Image comparison (side-by-side)

Image comparison is used to determine the opinion of a large group of people — what they like, what’s more convenient, and so on. For example, understanding which images in advertising banners people prefer, or which illustration prompts them to click on an article.

How to do it : When comparing two images, choose the one that is more suited to the given context, for example, when choosing which interface design is superior.

On Toloka : Compare two images and find out which one is better. Use for data verification or test user preferences with ads or website designs.

Object detection

An object detection algorithm detects an object in an image and its location in the image frame. The location is marked with different shapes. For example, facial recognition dots are used for models in facial recognition systems. Bounding box — the smallest rectangle that contains the entire object in the image — can be used to define the location of objects.

How to do it : You may be given the task of finding and highlighting an object in an image. For example, traffic lights, pedestrians, or cars, which can be used to train models in self-driving cars. A machine learning engineer selects a shape (rectangle, polygon, or other) that is best suited to the given model. Then the engineer assigns a specific task to you (the annotator ) such as “highlight all the traffic lights with rectangles”.

On Toloka : Supports bounding boxes, polygons, or keypoints, as well as image segmentation and tagging based on your own ontology. Use for computer vision applications.

Text recognition from images

Text recognition for images is used for various text recognition systems — think using your camera app to hover over text written in a foreign language and getting a translation.

How to do it : Generally, these tasks include images that contain text (such images are selected in advance). Your task is to read the text and type what is written there.

On Toloka : Identify and transcribe text in images. Train text recognition algorithms or validate and fine-tune the output of your OCR models.

With Toloka, you can control data labeling accuracy to build a predictable pipeline of high-quality training data that impacts your CV algorithms. Our platform supports annotation for image classification, semantic segmentation, object detection and recognition, and instance segmentation. Labeling tools include bounding boxes, polygons, and keypoint annotation.

Moreover, Toloka offers a ready-to-use object detection pipeline to get a human-labeled dataset for your images.

To sum up: what’s next

Image labeling and annotation helps improve computer vision accuracy by enabling object recognition. Annotated data is particularly important when the model is trying to solve a new field or domain. Training AI and machine learning with labels helps these models identify patterns until they can recognize objects on their own. This technology is increasingly being used across industries and is showing no signs of slowing down.

Moreover, Toloka provides a data labeling platform where millions of annotators from all around the world perform tasks posted by AI teams and companies. The platform brings these two audiences together, and its smart technologies transform the crowd into computing power. Toloka provides AI-powered businesses with the tools they need to manage the quality of data labeling and allows them to smoothly build a pipeline that delivers high-quality labeled data for machine learning.

The platform contains various annotation tools, including image labeling tools to train computer vision or image classification models. There, you can collect new data or label your own training data sets with relevant objects specifically for your project. You can make your own labeling instructions and set up the quality assurance process, or ask Toloka's engineers for help.

We invite you to browse through some of our step-by-step instructions and templates for different types of image labeling below:

Image comparison (Side-by-side)

Object recognition & detection

About Toloka

Toloka is a European company based in Amsterdam, the Netherlands that provides data for Generative AI development. Toloka empowers businesses to build high quality, safe, and responsible AI. We are the trusted data partner for all stages of AI development from training to evaluation. Toloka has over a decade of experience supporting clients with its unique methodology and optimal combination of machine learning technology and human expertise, offering the highest quality and scalability in the market.

Article written by:

Subscribe to Toloka News

Case studies, product news, and other articles straight to your inbox.

Back to top

Recent articles

View all articles

Breaking barriers: multilingual large language models in a globalized world

Sep 23, 2024

Toloka and Top Universities Launch Innovative Benchmark for Detecting AI-Generated Texts

Sep 17, 2024

LLM Alignment to human values and goals

Sep 9, 2024

More about Toloka

What is Toloka’s mission?

Where is Toloka located?

What is Toloka’s key area of expertise?

How long has Toloka been in the AI market?

How does Toloka ensure the quality and accuracy of the data collected?

How does Toloka source and manage its experts and AI tutors?

What types of projects or tasks does Toloka typically handle?

What industries and use cases does Toloka focus on?

Data Labeling Platform

Adaptive Auto ML

ML Platform

E-commerce Services

Search Relevance

Content Moderation

Computer Vision

NLP and Speech

Technologies

Global Crowd

Easy Integration

Security and Privacy

Success Stories

Toloka Research

Public Datasets

Open Source

Responsible AI

Education Partnerships

Partnerships

In the Media

Brand Guidelines

© 2024 Toloka AI BV

Manage cookies

Privacy Notice

Terms of Use

Code of Conduct

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Simplified labeling process for medical image segmentation

Affiliation.

• 1 CBIM Center, Rutgers University, Piscataway, NJ 08554, USA.
• PMID: 23286072
• DOI: 10.1007/978-3-642-33418-4_48

Image segmentation plays a crucial role in many medical imaging applications by automatically locating the regions of interest. Typically supervised learning based segmentation methods require a large set of accurately labeled training data. However, thel labeling process is tedious, time consuming and sometimes not necessary. We propose a robust logistic regression algorithm to handle label outliers such that doctors do not need to waste time on precisely labeling images for training set. To validate its effectiveness and efficiency, we conduct carefully designed experiments on cervigram image segmentation while there exist label outliers. Experimental results show that the proposed robust logistic regression algorithms achieve superior performance compared to previous methods, which validates the benefits of the proposed algorithms.

PubMed Disclaimer

Similar articles

• Segmentation of volumetric MRA images by using capillary active contour. Yan P, Kassim AA. Yan P, et al. Med Image Anal. 2006 Jun;10(3):317-29. doi: 10.1016/j.media.2005.12.002. Epub 2006 Feb 7. Med Image Anal. 2006. PMID: 16464631
• A run-based two-scan labeling algorithm. He L, Chao Y, Suzuki K. He L, et al. IEEE Trans Image Process. 2008 May;17(5):749-56. doi: 10.1109/TIP.2008.919369. IEEE Trans Image Process. 2008. PMID: 18390379
• Transferring color between three-dimensional objects. Shen HL, Xin JH. Shen HL, et al. Appl Opt. 2005 Apr 1;44(10):1969-76. doi: 10.1364/ao.44.001969. Appl Opt. 2005. PMID: 15813533
• Supervised learning of semantic classes for image annotation and retrieval. Carneiro G, Chan AB, Moreno PJ, Vasconcelos N. Carneiro G, et al. IEEE Trans Pattern Anal Mach Intell. 2007 Mar;29(3):394-410. doi: 10.1109/TPAMI.2007.61. IEEE Trans Pattern Anal Mach Intell. 2007. PMID: 17224611
• Text extraction and document image segmentation using matched wavelets and MRF model. Kumar S, Gupta R, Khanna N, Chaudhury S, Joshi SD. Kumar S, et al. IEEE Trans Image Process. 2007 Aug;16(8):2117-28. doi: 10.1109/tip.2007.900098. IEEE Trans Image Process. 2007. PMID: 17688216
• Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know. Yoon JT, Lee KM, Oh JH, Kim HG, Jeong JW. Yoon JT, et al. Diagnostics (Basel). 2024 Aug 13;14(16):1756. doi: 10.3390/diagnostics14161756. Diagnostics (Basel). 2024. PMID: 39202244 Free PMC article. Review.
• Search in MeSH

LinkOut - more resources

Other literature sources.

• The Lens - Patent Citations
• International Agency for Research on Cancer - Screening Group
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

Image Polygonal Annotation with Python (polygon, rectangle, circle, line, point and image-level flag annotation).

wkentaro/labelme

Folders and files.

Repository files navigation

Image polygonal annotation with python.

Description

Labelme is a graphical image annotation tool inspired by http://labelme.csail.mit.edu . It is written in Python and uses Qt for its graphical interface.

• Image annotation for polygon, rectangle, circle, line and point. ( tutorial )
• Image flag annotation for classification and cleaning. ( #166 )
• Video annotation. ( video annotation )
• GUI customization (predefined labels / flags, auto-saving, label validation, etc). ( #144 )
• Exporting VOC-format dataset for semantic/instance segmentation. ( semantic segmentation , instance segmentation )
• Exporting COCO-format dataset for instance segmentation. ( instance segmentation )

Starter Guide

If you're new to Labelme, you can get started with Labelme Starter (FREE), which contains:

• Installation guides for all platforms: Windows, macOS, and Linux 💻
• Step-by-step tutorials : first annotation to editing, exporting, and integrating with other programs 📕
• A compilation of valuable resources for further exploration 🔗.

Installation

There are options:

• Platform agnostic installation: Anaconda
• Platform specific installation: Ubuntu , macOS , Windows
• Pre-build binaries from the release section

You need install Anaconda , then run below:

Install Anaconda , then in an Anaconda Prompt run:

Run labelme --help for detail. The annotations are saved as a JSON file.

Command Line Arguments

• --output specifies the location that annotations will be written to. If the location ends with .json, a single annotation will be written to this file. Only one image can be annotated if a location is specified with .json. If the location does not end with .json, the program will assume it is a directory. Annotations will be stored in this directory with a name that corresponds to the image that the annotation was made on.
• The first time you run labelme, it will create a config file in ~/.labelmerc . You can edit this file and the changes will be applied the next time that you launch labelme. If you would prefer to use a config file from another location, you can specify this file with the --config flag.
• Without the --nosortlabels flag, the program will list labels in alphabetical order. When the program is run with this flag, it will display labels in the order that they are provided.
• Flags are assigned to an entire image. Example
• Labels are assigned to a single polygon. Example
• How to convert JSON file to numpy array? See examples/tutorial .
• How to load label PNG file? See examples/tutorial .
• How to get annotations for semantic segmentation? See examples/semantic_segmentation .
• How to get annotations for instance segmentation? See examples/instance_segmentation .
• Image Classification
• Bounding Box Detection
• Semantic Segmentation
• Instance Segmentation
• Video Annotation

How to develop

How to build standalone executable.

Below shows how to build the standalone executable on macOS, Linux and Windows.

How to contribute

Make sure below test passes on your environment. See .github/workflows/ci.yml for more detail.

Acknowledgement

This repo is the fork of mpitid/pylabelme .

Releases 68

Sponsor this project.

Contributors 57

• Python 99.8%
• Makefile 0.2%

labelImg 1.8.6

pip install labelImg Copy PIP instructions

Released: Oct 11, 2021

LabelImg is a graphical image annotation tool and label object bounding boxes in images

Verified details

Maintainers.

Unverified details

Project links.

• License: MIT License (MIT license)
• Author: TzuTa Lin
• Tags labelImg, labelTool, development, annotation, deeplearning
• Requires: Python >=3.0.0

Classifiers

• 5 - Production/Stable
• OSI Approved :: MIT License
• Python :: 3
• Python :: 3.3
• Python :: 3.4
• Python :: 3.5
• Python :: 3.6
• Python :: 3.7

Project description

LabelImg is a graphical image annotation tool.

It is written in Python and uses Qt for its graphical interface.

Annotations are saved as XML files in PASCAL VOC format, the format used by ImageNet . Besides, it also supports YOLO and CreateML formats.

Watch a demo video

Installation

Get from pypi but only python3.0 or above.

This is the simplest (one-command) install method on modern Linux distributions such as Ubuntu and Fedora.

Build from source

Linux/Ubuntu/Mac requires at least Python 2.6 and has been tested with PyQt 4.8 . However, Python 3 or above and PyQt5 are strongly recommended.

Ubuntu Linux

Python 3 + Qt5

Python 3 Virtualenv (Recommended)

Virtualenv can avoid a lot of the QT / Python version issues

Note: The Last command gives you a nice .app file with a new SVG Icon in your /Applications folder. You can consider using the script: build-tools/build-for-macos.sh

Install Python , PyQt5 and install lxml .

Open cmd and go to the labelImg directory

Windows + Anaconda

Download and install Anaconda (Python 3+)

Open the Anaconda Prompt and go to the labelImg directory

You can pull the image which has all of the installed and required dependencies. Watch a demo video

Steps (PascalVOC)

The annotation will be saved to the folder you specify.

You can refer to the below hotkeys to speed up your workflow.

Steps (YOLO)

A txt file of YOLO format will be saved in the same folder as your image with same name. A file named “classes.txt” is saved to that folder too. “classes.txt” defines the list of class names that your YOLO label refers to.

Create pre-defined classes

You can edit the data/predefined_classes.txt to load pre-defined classes

Verify Image:

When pressing space, the user can flag the image as verified, a green background will appear. This is used when creating a dataset automatically, the user can then through all the pictures and flag them instead of annotate them.

The difficult field is set to 1 indicates that the object has been annotated as “difficult”, for example, an object which is clearly visible but difficult to recognize without substantial use of context. According to your deep neural network implementation, you can include or exclude difficult objects during training.

How to reset the settings

In case there are issues with loading the classes, you can either:

rm ~/.labelImgSettings.pkl

How to contribute

Send a pull request

Free software: MIT license

Citation: Tzutalin. LabelImg. Git code (2015). https://github.com/tzutalin/labelImg

Related and additional tools

Stargazers over time.

1.8.6 (2021-10-10)

1.8.5 (2021-04-11), 1.8.4 (2020-11-04), 1.8.2 (2018-12-02), 1.8.1 (2018-12-02), 1.8.0 (2018-10-21), 1.7.0 (2018-05-18), 1.6.1 (2018-04-17), 1.6.0 (2018-01-29), 1.5.2 (2017-10-24), 1.5.1 (2017-9-27), 1.5.0 (2017-9-14), 1.4.3 (2017-08-09), 1.4.0 (2017-07-07), 1.3.4 (2017-07-07), 1.3.3 (2017-05-31), 1.3.2 (2017-05-18), 1.3.1 (2017-05-11), 1.3.0 (2017-04-22), 1.2.3 (2017-04-22), 1.2.2 (2017-01-09), project details, release history release notifications | rss feed.

Oct 11, 2021

Apr 12, 2021

Nov 15, 2020

May 26, 2019

Dec 3, 2018

Oct 22, 2018

May 19, 2018

Jan 29, 2018

Oct 24, 2017

Sep 27, 2017

Sep 14, 2017

Aug 9, 2017

Jul 21, 2017

Jul 7, 2017

May 31, 2017

May 18, 2017

May 11, 2017

May 3, 2017

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages .

Source Distribution

Uploaded Oct 11, 2021 Source

Hashes for labelImg-1.8.6.tar.gz

• português (Brasil)

Supported by

labeling the body

Get better grades with Learn

82% of students achieve A’s after using Learn

Medical Terminology for Health Professions

Precalculus

muscle Choose matching term 1 neur/o- 2 arthr/o- 3 myel/o 4 my/o- Don't know?

Category-Aware Dynamic Label Assignment with High-Quality Oriented Proposal

Objects in aerial images are typically embedded in complex backgrounds and exhibit arbitrary orientations. When employing oriented bounding boxes (OBB) to represent arbitrary oriented objects, the periodicity of angles could lead to discontinuities in label regression values at the boundaries, inducing abrupt fluctuations in the loss function. To address this problem, an OBB representation based on the complex plane is introduced in the oriented detection framework, and a trigonometric loss function is proposed. Moreover, leveraging prior knowledge of complex background environments and significant differences in large objects in aerial images, a conformer RPN head is constructed to predict angle information. The proposed loss function and conformer RPN head jointly generate high-quality oriented proposals. A category-aware dynamic label assignment based on predicted category feedback is proposed to address the limitations of solely relying on IoU for proposal label assignment. This method makes negative sample selection more representative, ensuring consistency between classification and regression features. Experiments were conducted on four realistic oriented detection datasets, and the results demonstrate superior performance in oriented object detection with minimal parameter tuning and time costs. Specifically, mean average precision (mAP) scores of 82.02%, 71.99%, 69.87%, and 98.77% were achieved on the DOTA-v1.0, DOTA-v1.5, DIOR-R, and HRSC2016 datasets, respectively.

Index Terms:

I introduction.

Oriented object detection is one of the challenging tasks in computer vision [ 1 , 2 , 3 ] , which aims to assign a bounding box with a unique semantic category label to each object in the given images [ 2 , 4 , 5 ] . Since these images are often captured from a bird’s-eye view, with objects typically arranged in dense rows in arbitrary directions against a complex background, researchers generally adopted oriented bounding boxes (OBB) to represent oriented objects more compactly. Numerous advanced orientation detectors have evolved by introducing an additional angular branch to traditional horizontal detectors, such as Faster R-CNN-O [ 6 ] and RetinaNet-O [ 7 ] , along with various variants that incorporate refined heads [ 8 , 9 , 10 ] .

Compared with horizontal detectors, oriented detectors typically need to output the angle of the object’s bounding boxes. The angle regression paradigm may potentially have at least two issues. The first issue concerns angle boundary discontinuities, and the second pertains to inconsistencies between the loss function and evaluation metrics. For instance, using the long-side definition method, the OBB angle ranges from [ − π 2 , π 2 ) 𝜋 2 𝜋 2 [-\frac{\pi}{2},\frac{\pi}{2}) [ - divide start_ARG italic_π end_ARG start_ARG 2 end_ARG , divide start_ARG italic_π end_ARG start_ARG 2 end_ARG ) , as shown in Fig. 1 (a). The green line represents the target angle at the angle boundary. The solid red line and dashed red line represent predicted angles near − π 2 𝜋 2 -\frac{\pi}{2} - divide start_ARG italic_π end_ARG start_ARG 2 end_ARG and π 2 𝜋 2 \frac{\pi}{2} divide start_ARG italic_π end_ARG start_ARG 2 end_ARG , respectively. They are equidistant from the target angle. The  smooth  ⁢ L 1  smooth  subscript 𝐿 1 \text{ smooth }L_{1} smooth italic_L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT loss is illustrated in Fig. 1 (b), where the loss at the red dashed line is much larger than at the red solid line. The 1 − IoU 1 IoU 1-\text{IoU} 1 - IoU loss is illustrated in Fig. 1 (c), where the loss at the red dashed line is equivalent to that at the red solid line. It can be seen that the IoU-based loss is an ideal evaluation that can solve the angle boundary discontinuity problem. However, the IoU-based loss for oriented bounding boxes is non-differentiable and cannot be backpropagated.

To address the aforementioned issue, the complex plane coordinates ( sin ⁡ θ , cos ⁡ θ ) 𝜃 𝜃 (\sin\theta,\cos\theta) ( roman_sin italic_θ , roman_cos italic_θ ) of the boundary-continuous periodic are utilized as replacements for the boundary-discontinuous periodic angle θ 𝜃 \theta italic_θ . A smooth and continuous IoU-like loss function is designed as follows:

Here, θ p subscript 𝜃 𝑝 \theta_{p} italic_θ start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT represents the predicted angle, and θ g subscript 𝜃 𝑔 \theta_{g} italic_θ start_POSTSUBSCRIPT italic_g end_POSTSUBSCRIPT represents the target angle. The relationship between the predicted values and the loss function is illustrated in Fig. 1 (d). The blue and magenta curves represent the cases with target angles of − π 2 𝜋 2 -\frac{\pi}{2} - divide start_ARG italic_π end_ARG start_ARG 2 end_ARG and 0 0 , respectively. It can be observed from Fig. 1 (d) that every target angle has two convergence points: one is the target angle itself, and the other is its symmetrical point in the complex plane. The loss function guides the predicted angle to converge towards the nearer convergence point. Fig. 1 (e) shows the top view of Fig. 1 (d). Compared with the long-side definition method in Fig. 1 (a), the proposed loss function automatically generates a symmetrical point in the complex plane, solving the boundary discontinuity problem. Fig. 1 (e) shows the side view of Fig. 1 (d). The effectiveness of our loss function is further illustrated by the similarity between Fig. 1 (f) and (b). Compared with the IoU-based loss, our loss function is differentiable.

The boundary problem of the angle is eliminated by improving the representation and the loss function. However, the regression of angle information is limited by the incompatibility between the classification and regression branches in the orientation detector. Additionally, there is a misalignment between the receptive field of the convolutional features and the OBB [ 11 ] , with different object types requiring different receptive fields [ 12 ] . A conformer RPN head is designed using the strong a priori knowledge inherent in remote sensing images, combined with vanilla convolution and multi-head self-attention mechanisms. This design can dynamically adjust the receptive fields, thereby mitigating the misclassification problem caused by the fixed receptive fields of the detector head and efficiently learning angle information.

The conformer RPN head and the trigonometric loss function generate high-quality proposals for the detector. Traditional IoU-based label assignment often overlooks the impact of the actual shape characteristics in proposals. It is particularly unreasonable to use the same preset IoU threshold for label assignment in oriented object detection. To enhance the rationality of label assignment, a category-aware dynamic label assignment method has been developed based on feedback category prediction information.

In summary, the contributions of this work are:

A new angular loss function is proposed by modeling angles in the complex plane, which fundamentally addresses the boundary problem. The method is flexible and simple enough to be integrated into any orientation detection framework.

A conformer RPN head that dynamically adjusts the reception field is proposed to effectively correlate classification and regression tasks.

Based on feedback category prediction, a category-aware dynamic label assignment method is designed to correct the irrationality of label assignment.

II RELATED WORK

With the advancement of deep learning, object detection has made significant progress in recent years. This section will first discuss the existing orientation detectors for both HBB and OBB based detection. Then, representative work on OBB representation, loss functions, and label assignment strategy will be presented.

II-A Oriented Object Detection

Early works on oriented object detection were generally derived from horizontal detectors. Due to objects in aerial images typically appearing at arbitrary orientations, they are affected by spatial misalignment between oriented objects and horizontal proposals. Liu [ 13 ] and Ma [ 14 ] address this issue by using rotating anchors with different angles, scales, and aspect ratios. Ding et al. [ 8 ] proposed a rotating region of interest (RRoI) learner that learns the parameters of the transformation of a horizontal region of interest (RoI) to RRoI under the supervision of OBB. Yang et al. [ 9 ] presented a refined box with aligned features by reconstructing the feature map. In contrast to Yang et al [ 9 ] , S 2 ⁢ A-Net superscript S 2 A-Net \text{S}^{2}\text{A-Net} S start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT A-Net [ 10 ] generates orientation detection results by cleverly using deformable convolution to self-adaptively align deep features. However, these methods require complex RPN operations and extensive boundary box transformation computations. Additionally, some approaches [ 15 , 12 , 16 ] has focused on developing high-quality object detectors by enhancing the backbone network. While the aforementioned methods contribute to improving detection performance, they still encounter challenges such as the boundary problem and the label assignment problem.

II-B Different Representations for Oriented bounding Boxes

Recent work has typically focused on developing novel OBB representations to alleviate boundary problems. He et al. [ 17 ] and Liao et al. [ 18 ] used the corner points of oriented rectangles to represent bounding boxes. Gliding Vertex [ 19 ] introduced four length ratios to form a new representation. Oriented R-CNN [ 20 ] used parallelogram midpoint offsets to simplify the representation to six-parameter. Yao et al. [ 21 ] designed a five-parameter representation using the circumscribed circle of a rectangular box. Despite avoiding directly using the angles from oriented bounding boxes, these methods require complex transformation calculations and post-processing, such as regularization. In addition to anchor-based methods, anchor-free and keypoint-based methods have also emerged. Chen et al. [ 22 ] applied the IoU loss function for OBB based on pixel statistics to anchor-free methods. Oriented RepPoints [ 23 ] use adaptive point sets to capture the geometric structure of oriented objects. OSKDet [ 24 ] explores a novel disordered keypoint heatmap fusion method to learn the shape and direction of the oriented object. Cheng et al. [ 25 ] designed AOPG to generate high-quality proposals. However, these methods require longer training costs to achieve the same results as frame-based methods, and there is semantic ambiguity in the object center point.

II-C Loss Functions

In addition to using different OBB representation methods, some work attempts to address angle boundary problem through loss functions. Qian et al. [ 26 ] proposed a modulated rotation loss in RSDet to dismiss loss discontinuities and discussed regression inconsistency. However, RSDet [ 26 ] is only a remedial measure taken after the problem is discovered, rather than designing a prediction method that avoids boundary issues. Yang et al. advanced works [ 27 , 28 , 29 ] , proposed to convert the angle regression problem to a classification task and use circular smooth labels or Densely Coded Labels to replace the one-hot labels, get rid of the boundary discontinuity issue. Wang et al. proposed Gaussian Focal Loss [ 30 ] as a more effective alternative to the classification-based rotation detectors. Yang et al. [ 31 ] design a novel IoU constant factor to alleviate the boundary problem, GWD [ 32 ] and KLD [ 33 ] transform an OBB into a 2-D Gaussian distribution and solve the derivation of RIoU problem by computing the distance between the distributions. But the price to pay is the presence of theoretical errors and the increased number of model parameters and calculations.

II-D Label Assignment Strategy

Label assignment is a critical component in selecting high-quality samples for the detector. The sampling issue for horizontal detectors has been more extensively researched. For example, focal loss [ 7 ] weights the samples using a reconstruction loss function. ATSS [ 34 ] automatically classifies positive and negative training samples based on the statistical properties of the object. OTA [ 35 ] formulates the sample allocation process as an optimal transmission problem. OHEM [ 36 ] mines some hard example samples for training. PISA [ 37 ] assessed the sample difficulty based on mean average precision (mAP). However, there are fewer studies on region sample selection for oriented detectors. Hou [ 38 ] added a shape-adaptive strategy to the label assignment. Huang [ 39 ] proposed a Gaussian heat map label assignment method. Yu [ 11 ] proposed a soft label assignment method. Although these methods have designed dynamic label assignment thresholds, they lack a connection between the detection results and the label assignment.

III METHODS

Iii-a overview.

This section introduces an improved two-stage oriented object detection framework, as illustrated in Fig. 2 . In the RPN stage, the trigonometric loss function and a conformer RPN head are proposed to generate high-quality oriented proposals. In the ROI stage, a category-aware dynamic label assignment is used to rationally select negative samples. To streamline the discussion related to Fig. 2 , the recommended approach will be elaborated on in the subsequent sections.

III-B Trigonometric Loss Function (TLF)

According to the analysis in the introduction, the OBB is represented using a six-element tuple ( x , y , w , h , sin ⁡ θ , cos ⁡ θ ) 𝑥 𝑦 𝑤 ℎ 𝜃 𝜃 (x,y,w,h,\sin\theta,\cos\theta) ( italic_x , italic_y , italic_w , italic_h , roman_sin italic_θ , roman_cos italic_θ ) . Here ( x , y ) 𝑥 𝑦 (x,y) ( italic_x , italic_y ) are the coordinates of the center, w 𝑤 w italic_w is the length of the long side, h ℎ h italic_h is the length of the short side, and θ 𝜃 \theta italic_θ is the angle between w 𝑤 w italic_w and the x-axis. The OBB regression loss function for RPN training is designed as follows:

where the parameters 𝒕 = ( t x , t y , t w , t h , t sin ⁡ θ , t cos ⁡ θ ) 𝒕 subscript 𝑡 𝑥 subscript 𝑡 𝑦 subscript 𝑡 𝑤 subscript 𝑡 ℎ subscript 𝑡 𝜃 subscript 𝑡 𝜃 \boldsymbol{t}=(t_{x},~{}t_{y},~{}t_{w},~{}t_{h},~{}t_{\sin\theta},~{}t_{\cos% \theta}) bold_italic_t = ( italic_t start_POSTSUBSCRIPT italic_x end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT italic_y end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT italic_w end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT italic_h end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT roman_sin italic_θ end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT roman_cos italic_θ end_POSTSUBSCRIPT ) and 𝒕 𝒑 = ( t x p , t y p , t w p , t h p , t sin ⁡ θ p , t cos ⁡ θ p ) superscript 𝒕 𝒑 superscript subscript 𝑡 𝑥 𝑝 superscript subscript 𝑡 𝑦 𝑝 superscript subscript 𝑡 𝑤 𝑝 superscript subscript 𝑡 ℎ 𝑝 superscript subscript 𝑡 𝜃 𝑝 superscript subscript 𝑡 𝜃 𝑝 \boldsymbol{t^{p}}=(t_{x}^{p},~{}t_{y}^{p},~{}t_{w}^{p},~{}t_{h}^{p},~{}t_{% \sin\theta}^{p},~{}t_{\cos\theta}^{p}) bold_italic_t start_POSTSUPERSCRIPT bold_italic_p end_POSTSUPERSCRIPT = ( italic_t start_POSTSUBSCRIPT italic_x end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT , italic_t start_POSTSUBSCRIPT italic_y end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT , italic_t start_POSTSUBSCRIPT italic_w end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT , italic_t start_POSTSUBSCRIPT italic_h end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT , italic_t start_POSTSUBSCRIPT roman_sin italic_θ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT , italic_t start_POSTSUBSCRIPT roman_cos italic_θ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT ) denote the offsets of the ground truth and predicted proposal relative to the anchor box, respectively. The variables ( x , y , w , h ) 𝑥 𝑦 𝑤 ℎ (x,y,w,h) ( italic_x , italic_y , italic_w , italic_h ) use  smooth  ⁢ L 1  smooth  subscript 𝐿 1 \text{ smooth }L_{1} smooth italic_L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT loss, while θ 𝜃 \theta italic_θ use the L ⁢ ( t θ , t θ p ) 𝐿 subscript 𝑡 𝜃 superscript subscript 𝑡 𝜃 𝑝 L{(t_{\theta},t_{\theta}^{p})} italic_L ( italic_t start_POSTSUBSCRIPT italic_θ end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT italic_θ end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT ) loss as defined in equation ( 5 ).

The offsets 𝒕 𝒕 \boldsymbol{t} bold_italic_t are calculated from the ground truth and anchor box. The encoding equation is as follows:

The proposals are calculated based on the offsets 𝒕 𝒑 superscript 𝒕 𝒑 \boldsymbol{t^{p}} bold_italic_t start_POSTSUPERSCRIPT bold_italic_p end_POSTSUPERSCRIPT and anchor boxes. This decoding equation is as follows:

During the encoding and decoding process, it is common to swap the width and height when the angle between the prior and the target exceeds π 4 𝜋 4 \frac{\pi}{4} divide start_ARG italic_π end_ARG start_ARG 4 end_ARG . Although this reduces the extreme values of the angle loss, it relies on the precise prediction of the width and height. The application of trigonometric functions in the encoding, decoding, and loss functions completely resolves the angle boundary problem. Consequently, our encoding and decoding process has removed the swap width and height operation, effectively preventing the influence between the model’s angle predictions and width and height predictions.

The introduction only analyzes the relationship between predicted angles and IoU under fixed aspect ratios. Objects with different aspect ratios exhibit distinct sensitivities to angles and different curves of IoU versus angular deviation, as shown in Fig. 3 . To address this problem, an aspect ratio sensitivity factor w h 𝑤 ℎ \sqrt{\frac{w}{h}} square-root start_ARG divide start_ARG italic_w end_ARG start_ARG italic_h end_ARG end_ARG is added to equation ( 1 ). The loss in the equation ( 2 ) angular branch is calculated as follows.

Compared to other methods, TLF adopts complex-plane coordinates instead of angles for a consistent representation, encoding, decoding, and loss functions. The application of TLF in the model ensures high consistency between training and evaluation metrics, offering an ideal solution for angular boundary problems. Experiments show that our method is more stable during training (see Fig. 9 ), and the results are superior to those of other methods (see Table I ). In the RetinaNet-O experiment, the regression loss for OBB replaces  smooth  ⁢ L 1  smooth  subscript 𝐿 1 \text{ smooth }L_{1} smooth italic_L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT with L 1 ⁢ ( x ) = | x | subscript 𝐿 1 𝑥 𝑥 L_{1}(x)=\left|x\right| italic_L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT ( italic_x ) = | italic_x | in equation ( 2 ).

III-C Conformer RPN Head

In a regular RPN head, the same convolution kernel is used for all channels of all input images. Due to the complex background environments and objective differences in remote sensing images, recognizing different types of objects requires varying receptive fields. To efficiently handle various ranges of detected objects, a multi-head self-attention mechanism is proposed to complement the conventional convolutional approach. A conformer RPN head containing vanilla convolution, dilation convolution, and multi-head self-attention mechanisms, as shown in Fig. 4 . The incorporation of multi-head self-attention enables dynamic adjustment of the receptive field for feature extraction.

Given the input feature X : ( N , C i ⁢ n , H , W ) : 𝑋 𝑁 subscript 𝐶 𝑖 𝑛 𝐻 𝑊 X:(N,C_{in},H,W) italic_X : ( italic_N , italic_C start_POSTSUBSCRIPT italic_i italic_n end_POSTSUBSCRIPT , italic_H , italic_W ) . For each location p 𝑝 p italic_p on the output of vanilla convolution and dilation convolution can be precisely described as:

where the grid ℛ ℛ \mathcal{R} caligraphic_R defines the convolution kernel size and dilation, 𝒑 n subscript 𝒑 𝑛 \boldsymbol{p}_{n} bold_italic_p start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT enumerates the locations in ℛ ℛ \mathcal{R} caligraphic_R .

For each location p 𝑝 p italic_p on the output of multi-head self-attention can be precisely described as:

where N h subscript 𝑁 ℎ N_{h} italic_N start_POSTSUBSCRIPT italic_h end_POSTSUBSCRIPT denotes the number of heads in multi-head self-attention, S ⁢ A ⁢ ( 𝑿 ) = s ⁢ o ⁢ f ⁢ t ⁢ m ⁢ a ⁢ x ⁢ ( Q ⁢ K T d k ) ⁢ V 𝑆 𝐴 𝑿 𝑠 𝑜 𝑓 𝑡 𝑚 𝑎 𝑥 𝑄 superscript 𝐾 𝑇 subscript 𝑑 𝑘 𝑉 {SA}\boldsymbol{(X)}=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V italic_S italic_A bold_( bold_italic_X bold_) = italic_s italic_o italic_f italic_t italic_m italic_a italic_x ( divide start_ARG italic_Q italic_K start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT end_ARG start_ARG square-root start_ARG italic_d start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT end_ARG end_ARG ) italic_V , and the superscript ( h ) ℎ (h) ( italic_h ) denotes the h ℎ h italic_h -th head. In practice, the S ⁢ A 𝑆 𝐴 {SA} italic_S italic_A and 𝑾 ( h ) superscript 𝑾 ℎ \boldsymbol{W}^{(h)} bold_italic_W start_POSTSUPERSCRIPT ( italic_h ) end_POSTSUPERSCRIPT of each head are independent of each other and do not share weights, and the result is obtained by sequentially splicing and then directly cross-multiplying them.

Since vanilla convolution and dilation volumes with fixed and horizontal receptive domains cannot align oriented objects, while multi-head self-attention can capture the global dependency and consider the information of the whole input sequence. Therefore, in this approach, 1/4 channels of features are obtained using vanilla convolution and dilation convolution each, and another 1/2 channels of features are acquired using multiple self-attention mechanisms. These features are then combined to create a new feature extraction layer that dynamically adjusts the receptive field. Considering that conformer RPN head can perceive information about various objects, they are utilized in the current detector to extract classification and regression features. Finally, classification and regression results are obtained by using vanilla convolution.

III-D Class-Aware Dynamic Label Assignment (CDLA)

In object detection frameworks, the selection of positive and negative samples is crucial to the performance of the detector. Traditional label assignments based on IoU use a predefined IoU threshold for all objects. As illustrated by the maximum IoU label assignment in Fig. 5 , samples are divided into either negative or positive. However, relying solely on the IoU threshold to assign positive and negative samples is unreasonable when the IoU is between 0.4 and 0.5. As shown in Fig. 6 , the three oriented proposals have similar IoU values. Compared with the oriented proposal in Fig. 6 (a), which covers the main part of the object, Fig. 6 (b) contains more background information. Therefore, it is reasonable that the object class score in Fig. 6 (a) is 0.81, and the background class score in Fig. 6 (b) is 0.56. Compared with the IoU-based assignment that assigns the two proposals as negative samples, the proposed CDLA ignores these two proposals. Only when the object class score and the background class score are both less than 0.5, the proposed CDLA assigns the proposal as a negative sample, as shown in Fig. 6 (c). In summary, the proposed method achieves weakly supervised label assignment when the proposal IoU is between 0.4 and 0.5. If the classification is correct, either as the object class or background, it should be ignored during training. Otherwise, it would be assigned as a negative sample.

Conversely, the proposed method employs strong supervised label assignment when the proposal IoU is in the range of [ 0 , 0.3 ] 0 0.3 [0,0.3] [ 0 , 0.3 ] and the predicted background class score is less than 0.5. As indicated by the red boxes in Fig. 5 , these proposals are assigned as focus negative samples. This assignment strategy enables the detector to prioritize these negative samples during the training process.

As shown in Fig. 2 , the predicted categorical information by the ROI network is feedback to the ROI sampling module during training. A more representative category-aware dynamic label assignment was designed by integrating categorical information into the classical maximum IoU labeling assignment strategy. To clarify the CDLA algorithm, we summarize the process of label assignment and sample selection in Algorithm 1 . Firstly, calculate the RIOU for each ground truth and proposal in the image. Secondly, the feedback category information is used to perform a detailed division of the label assignment strategy. Finally, sample selection is performed using the data obtained from the aforementioned label assignment. Through these operations, we can achieve more flexible and reliable sampling strategies to ensure consistency between classification and regression features.

IV EXPERIMENTS AND RESULTS

Iv-a datasets.

We selected four public datasets for evaluating oriented object detection, namely DOTA-v1.0 [ 2 ] , DOTA-v1.5 [ 2 ] , DIOR-R [ 4 , 25 ] , and HRSC2016 [ 5 ] . Details are as follows.

IV-A 1 DOTA

The DOTA-v1.0 dataset is a comprehensive dataset designed for oriented object detection in aerial images. It comprises 2806 images captured by different sensors and platforms, with image sizes varying from 800 × 800 800 800 800\times 800 800 × 800 to 4000 × 4000 4000 4000 4000\times 4000 4000 × 4000 pixels. These images are divided into 1024 × 1024 1024 1024 1024\times 1024 1024 × 1024 sub-images with an overlap of 200 pixels. The fully annotated DOTA-v1.0 benchmark contains 188 282 instances across 15 common object categories, each labeled with an arbitrary quadrilateral. All reported results for DOTA were obtained by evaluating the test set on the official evaluation server.

Compared to DOTA-v1.0, DOTA-v1.5 introduces a new category called Container Crane (CC) and increases the number of micro-instances with a size smaller than 10 pixels. It contains a total of 403 318 instances.

IV-A 2 DIOR-R

DIOR-R consists of 23 463 images and 192 518 instances, covering 20 object classes. The dataset contains fixed-size images of 800 × 800 800 800 800\times 800 800 × 800 pixels, featuring a large variation in object sizes (ranging from 0.5 to 30 meters) in terms of spatial resolution, as well as significant inter-class and intra-class size variability. The training and validation sets contain a total of 11 725 images and 68 073 instances, while the testing set comprises 11 738 images and 124 445 instances.

IV-A 3 HRSC2016

HRSC2016 is a dataset for arbitrary-oriented ship detection in aerial images, containing 1061 images from six famous harbors. The image sizes range from 300 × 300 300 300 300\times 300 300 × 300 to 1500 × 900 1500 900 1500\times 900 1500 × 900 pixels, and we resize them to 800 × 800 800 800 800\times 800 800 × 800 pixels. The combined training set (436 images) and validation set (181 images) are used for training, while the remaining 455 images are used for testing. For detection accuracy on HRSC2016, we adopt the mAP as the evaluation criterion, consistent with PASCAL VOC 2007 and VOC 2012.

IV-B Implementation Details

All experiments were implemented using a single NVIDIA RTX 3090 with a batch size of 2, trained with half-precision on the MMRotate toolbox [ 40 ] . The training schedule uses “ 1 × 1\times 1 × ” for the DOTA and DIOR-R datasets, and “ 3 × 3\times 3 × ” for the HRSC2016 dataset. Competitive results have already been achieved using ResNet as the backbone network. Further validation was performed with ConvNeXt [ 41 ] to demonstrate the generalization and scalability of the framework. When using ResNet, the entire network was optimized using the SGD algorithm with a learning rate of 0.005, momentum of 0.9, weight decay of 0.0001, and a learning rate warm-up method with 500 iterations. When using ConvNeXt-T, the network was optimized using the AdamW algorithm with a learning rate of 0.0001, weight decay of 0.05, and a learning rate warm-up method with 1000 iterations.

IV-C Comparisons With State-of-the-Art Methods

Iv-c 1 results on dota.

Table I presents a comparison of our method with the SOTA methods using ResNet as the backbone on the DOTA-v1.0 dataset. RetinaNet-O with ResNet-50 as the backbone, using only TLF, achieves 71.19% mAP, surpassing many other outstanding loss functions. Using ResNet-50 as the backbone network, our method achieved 77.19% and 81.11% mAP for single-scale and multi-scale, respectively. In the single-scale setting, it surpassed the advanced Oriented R-CNN and ReDet with the mAP gains of 1.32% and 0.94%, respectively. Table II presents a comparison of our method using ConvNeXt-T as the backbone with other SOTA methods on the DOTA-v1.0 dataset. Our method achieves a SOTA result of 79.19% mAP for single-scale. In the multiscale setting, the mAP reached 82.02%, surpassing the SOTA results achieved with CNN as the backbone. We visualized some qualitative detection results, as shown in Fig. 7 .

Additionally, comparative experiments were conducted on the new version of the DOTA-v1.5 dataset. Table III lists the quantitative results of DOTA-v1.5. In the single-scale setting, our proposed method achieves 67.82% and 71.99% mAP using ResNet-50 and ConvNeXt-T as the backbone networks, respectively. Faced with challenging new situations, our method demonstrated satisfactory results, proving its robustness.

IV-C 2 Results on the DIOR-R

DIOR-R is a dataset annotated with five-parameter, and the results from this dataset are more indicative of the effectiveness of our method. The results of eight oriented detectors are reported in IV . Our proposed method achieves 65.93% and 69.87% mAP using ResNet-50 and ConvNeXt-T as the backbone networks, respectively. From the evaluation metrics and visualization results, it can be observed that our method achieved good performance, particularly on objects with large aspect ratios. Compared to other directed object detectors, our method shows strong competitiveness. Fig. 8 visualizes some detection results from the DIOR-R dataset.

IV-C 3 Results on HRSC2016

HRSC2016 contains numerous ship images with large aspect ratios in arbitrary directions, and the angle’s impact on IoUs is significant. The experimental results using RetinaNet-O as the baseline are presented in Table V . The experimental results for SOTA are presented in Table VI . Our method not only achieved improvements in mAP50 but also demonstrated significantly higher accuracy in mAP75 and mAP50:95 compared to similar methods. Specifically, using only our TLF on RetinaNet-O resulted in an mAP50 of 86.70%, mAP75 of 72.90%, and mAP50:95 of 61.34%. In the two-stage approach, our method achieves results of 90.70% and 90.89% on VOC2007, and 98.02% and 98.77% on VOC2012, respectively, using R-50 and ConvNeXt-T [ 41 ] backbone networks. Our method achieves SOTA results.

IV-D Ablation Studies

The DOTA-v1.0 dataset was used for ablation studies. In these experiments, Faster R-CNN [ 6 ] with ResNet50 [ 46 ] as the backbone was employed as the baseline method. As shown in Table VII , the baseline method achieves an mAP of 73.40%. By modifying the RPN with OBB, we established a new baseline, achieving an mAP of 74.35%. We used a separate prior for each innovation point to ensure a fair comparison. The step-by-step improvement in mAP validated the effectiveness of each design. The three innovations proposed in this paper resulted in a total improvement of 2.84% in mAP.

IV-D 1 Effectiveness of Loss

As shown in Table VII , using the TLF module solely on the baseline improved the mAP value by 1.83%, achieving an mAP of 76.18%. We also demonstrate the bounding box loss and iteration of Oriented R-CNN and our method in Fig. 9 , showing that our loss function exhibits strong stability.

IV-D 2 Effectiveness of Conformer RPN Head

The conformer RPN head module needs to be used based on the TLF. As shown in Table VII , the integration of the conformer RPN head module increases the mAP by 0.5% in the third row compared to the first row, and by 0.56% in the fifth row compared to the fourth row. The results indicate that the combination of convolution and multi-head self-attention helps in better capturing the correct classification information and sine-cosine components of the object angles. The conformer RPN head and TLF together contribute to generating high-quality proposals for ROI. Visualization comparison with Oriented R-CNN is shown in Fig. 10 .

IV-D 3 Effectiveness of CDLA

As shown in Table VII , using only the CDLA module increased the mAP on the baseline by 1.99%. Adding the CDLA module to the TLF resulted in a 0.45% increase in mAP. Furthermore, incorporating the CDLA module on top of both the TLF and conformer RPN head increased the mAP by 0.51%. The above results indicate that dynamically adjusting negative samples through category feedback facilitates flexible and reliable sample classification, which is more advantageous for network learning.

V CONCLUSION

This study effectively addresses the issue of inconsistent parameter regression and boundary problems by designing TLF through angle regression analysis on the complex plane. This loss function offers sufficient flexibility to be integrated into any oriented detection framework. To better enable detectors to learn the complex plane coordinates of angles, a conformer RPN head is designed. Improvements in the loss function and consistent RPN header generate high-quality oriented proposals. To fully leverage high-quality proposals, a category-aware dynamic label assignment method based on predicted category feedback is proposed. Experimental results demonstrate that this work achieves highly competitive performance on four well-known remote sensing benchmark datasets.

• [1] Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” Proc. IEEE , vol. 111, no. 3, pp. 257–276, Mar. 2023.
• [2] G.-S. Xia et al. , “DOTA: A large-scale dataset for object detection in aerial images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2018, pp. 3974–3983.
• [3] X. X. Zhu et al. , “Deep learning in remote sensing,” IEEE Geosci. Remote Sens. Mag. , vol. 5, no. 4, pp. 8–36, Dec. 2017.
• [4] K. Li, G. Wan, G. Cheng, L. Meng, and J. Han, “Object detection in optical remote sensing images: A survey and a new benchmark,” ISPRS-J. Photogramm. Remote Sens. , vol. 159, pp. 296–307, Jan. 2020.
• [5] Z. Liu, L. Yuan, L. Weng, and Y. Yang, “A high resolution optical satellite image dataset for ship recognition and some new baselines,” in Proc. Int. Conf. Pattern Recognit. Appl. Methods , 2017, pp. 324–331.
• [6] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 39, no. 6, pp. 1137–1149, Jun. 2017.
• [7] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense object detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , Oct. 2017, pp. 2999–3007.
• [8] J. Ding, N. Xue, Y. Long, G.-S. Xia, and Q. Lu, “Learning roi transformer for oriented object detection in aerial images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2019, pp. 2844–2853.
• [9] X. Yang, J. Yan, Z. Feng, and T. He, “R 3 det: Refined single-stage detector with feature refinement for rotating object,” in Proc. AAAI Conf. Artif. Intell. , vol. 35, 2021, pp. 3163–3171.
• [10] J. Han, J. Ding, J. Li, and G.-S. Xia, “Align deep features for oriented object detection,” IEEE Trans. Geosci. Remote Sensing , vol. 60, p. 5602511, 2022.
• [11] Y. Yu, X. Yang, J. Li, and X. Gao, “Object detection for aerial images with feature enhancement and soft label assignment,” IEEE Trans. Geosci. Remote Sensing , vol. 60, p. 5624216, 2022.
• [12] Y. Li, Q. Hou, Z. Zheng, M.-M. Cheng, J. Yang, and X. Li, “Large selective kernel network for remote sensing object detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , 2023, pp. 16 748–16 759.
• [13] Z. Liu, H. Wang, L. Weng, and Y. Yang, “Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds,” IEEE Geosci. Remote Sens. Lett. , vol. 13, no. 8, pp. 1074–1078, Aug. 2016.
• [14] J. Ma et al. , “Arbitrary-oriented scene text detection via rotation proposals,” IEEE Trans. Multimedia , vol. 20, no. 11, pp. 3111–3122, Nov. 2018.
• [15] J. Han, J. Ding, N. Xue, and G.-S. Xia, “Redet: A rotation-equivariant detector for aerial object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2021, pp. 2785–2794.
• [16] X. Cai, Q. Lai, Y. Wang, W. Wang, Z. Sun, and Y. Yao, “Poly kernel inception network for remote sensing detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2024.
• [17] W. He, X.-Y. Zhang, F. Yin, and C.-L. Liu, “Multi-oriented and multi-lingual scene text detection with direct regression,” IEEE Trans. Image Process. , vol. 27, no. 11, pp. 5406–5419, Nov. 2018.
• [18] M. Liao, B. Shi, and X. Bai, “Textboxes plus plus: A single-shot oriented scene text detector,” IEEE Trans. Image Process. , vol. 27, no. 8, pp. 3676–3690, Aug. 2018.
• [19] Y. Xu et al. , “Gliding vertex on the horizontal bounding box for multi-oriented object detection,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 43, no. 4, pp. 1452–1459, Apr. 2021.
• [20] X. Xie, G. Cheng, J. Wang, X. Yao, and J. Han, “Oriented R-CNN for object detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , 2021, pp. 3500–3509.
• [21] Y. Yao et al. , “On improving bounding box representations for oriented object detection,” IEEE Trans. Geosci. Remote Sensing , vol. 61, p. 5600111, 2023.
• [22] Z. Chen et al. , “PIoU loss: Towards accurate oriented object detection in complex environments,” in Proc. Eur. Conf. Comput. Vis. (ECCV) , vol. 12350, 2020, pp. 195–211.
• [23] W. Li, Y. Chen, K. Hu, and J. Zhu, “Oriented reppoints for aerial object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2022, pp. 1819–1828.
• [24] D. Lu, D. Li, Y. Li, and S. Wang, “OSKDet: Orientation-sensitive keypoint localization for rotated object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2022, pp. 1172–1182.
• [25] G. Cheng et al. , “Anchor-free oriented proposal generator for object detection,” IEEE Trans. Geosci. Remote Sensing , vol. 60, p. 5625411, 2022.
• [26] W. Qian, X. Yang, S. Peng, J. Yan, and Y. GuO, “Learning modulated loss for rotated object detection,” in Proc. AAAI Conf. Artif. Intell. , vol. 35, 2021, pp. 2458–2466.
• [27] X. Yang and J. Yan, “Arbitrary-oriented object detection with circular smooth label,” in Proc. Eur. Conf. Comput. Vis. (ECCV) , vol. 12353, 2020, pp. 677–694.
• [28] X. Yang, L. Hou, Y. Zhou, W. Wang, and J. Yan, “Dense label encoding for boundary discontinuity free rotation detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2021, pp. 15 814–15 824.
• [29] X. Yang and J. Yan, “On the arbitrary-oriented object detection: Classification based approaches revisited,” Int. J. Comput. Vis. , vol. 130, no. 5, pp. 1340–1365, May 2022.
• [30] J. Wang, F. Li, and H. Bi, “Gaussian focal loss: Learning distribution polarized angle prediction for rotated object detection in aerial images,” IEEE Trans. Geosci. Remote Sensing , vol. 60, p. 4707013, 2022.
• [31] X. Yang, J. Yan, W. Liao, X. Yang, J. Tang, and T. He, “SCRDet++: Detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 45, no. 2, pp. 2384–2399, Feb. 2023.
• [32] X. Yang, J. Yan, Q. Ming, W. Wang, X. Zhang, and Q. Tian, “Rethinking rotated object detection with gaussian wasserstein distance loss,” in Proc. Int. Conf. Mach. Learn. , vol. 139, 2021.
• [33] X. Yang et al. , “Learning high-precision bounding box for rotated object detection via kullback-leibler divergence,” in Proc. Adv. Neural Inf. Process. Syst. , vol. 34, 2021.
• [34] S. Zhang, C. Chi, Y. Yao, Z. Lei, and S. Z. Li, “Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Jun. 2020, pp. 9756–9765.
• [35] Z. Ge, S. Liu, Z. Liu, O. Yoshie, and J. Sun, “OTA: Optimal transport assignment for object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2021, pp. 303–312.
• [36] A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2016, pp. 761–769.
• [37] Y. Cao, K. Chen, C. C. Loy, and D. Lin, “Prime sample attention in object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Jun. 2020, pp. 11 580–11 588.
• [38] L. Hou, K. Lu, J. Xue, and Y. Li, “Shape-adaptive selection and measurement for oriented object detection,” in Proc. AAAI Conf. Artif. Intell. , 2022, pp. 923–932.
• [39] Z. Huang, W. Li, X.-G. Xia, and R. Tao, “A general gaussian heatmap label assignment for arbitrary-oriented object detection,” IEEE Trans. Image Process. , vol. 31, pp. 1895–1910, 2022.
• [40] Y. Zhou et al. , “MMRotate: A rotated object detection benchmark using pytorch,” in Proc. ACM Int. Conf. Multimedia , Oct. 2022, pp. 7331–7334.
• [41] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2022, pp. 11 966–11 976.
• [42] Y. Yu and F. Da, “Phase-shifting coder: Predicting accurate orientation in oriented object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2023, pp. 13 354–13 363.
• [43] G. Cheng et al. , “Dual-aligned oriented detector,” IEEE Trans. Geosci. Remote Sensing , vol. 60, p. 5618111, 2022.
• [44] Y. Pu et al. , “Adaptive rotated convolution for rotated object detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , 2023, pp. 6566–6577.
• [45] C. Xu et al. , “Dynamic coarse-to-fine learning for oriented tiny object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2023, pp. 7318–7328.
• [46] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2016, pp. 770–778.

• For educators
• English (US)
• English (India)
• English (UK)
• Greek Alphabet

This problem has been solved!

You'll get a detailed solution from a subject matter expert that helps you learn core concepts.

Question: Ex. 11: Best of Homework - Articulations and Body Movements Exercise 11 Review Sheet Art-labeling Activity 2 Femur medial meniscus lateral meniscus tibial collateral ligament patella 1010 tibia fibular collateral ligament fibula anterior cruciate ligament lateral condyle of the femur patellar ligament

really need the answer ASAP

Following is the labelling of the diagram :-

Not the question you’re looking for?

Post any question and get expert help quickly.