Python extension development with the Rust plugin [Beta][Pro]

Work seamlessly with Python projects that leverage Rust modules to speed up performance-critical components.

debugpy as the default debugger

Following its introduction as an optional backend in 2026.1, debugpy is now enabled by default for all Python projects and Jupyter notebooks, using the Debug Adapter Protocol (DAP).

Support for uv-backed tools and uvx

PyCharm now leverages the uv toolchain to streamline how you run your external development utilities, eliminating manual package setups that clutter your local environment.

Support for uv, Poetry, and Hatch multi-projects and uv workspaces [Beta]

Previously available as an optional feature in PyCharm 2026.1.1, this functionality is enabled by default in version 2026.2. It streamlines your subproject management and provides richer dependency insights directly within your configuration files. 

Editor minimap

Navigate complex source files and notebooks more efficiently with the official editor minimap. It provides a high-level visual overview of your document structure across all supported file types – while offering a dedicated layout built just for Jupyter notebooks.

Pyrefly type engine integration

Use Pyrefly as an external type engine to significantly accelerate code insight features for large-scale Python codebases.

Start new projects with AI

If you have a JetBrains AI license, you can now generate fully configured, runnable projects from scratch using natural language prompts directly from the Welcome screen.

Agent skills manager

AI agents are only as useful as the context they have. When they don’t have knowledge of your frameworks, conventions, and tooling, you end up re-explaining the same setup in every new chat window.

Agent skills fix that. Install them once in PyCharm, and your agents carry that domain knowledge across every project and session – automatically. Browse and manage skills directly from the IDE, expand the built-in library with external registries like public GitHub repositories, or let PyCharm import skills you’ve already set up for Claude Code or Codex. 

In this guide, we’ll examine the top object detection models available in 2026, comparing their architectures, performance characteristics, and ideal use cases to help you determine which models are best suited to your applications. 

Whether you’re building real-time video analytics, high-precision inspection systems, or resource-constrained edge applications, you’ll find clear guidance on which model best fits your requirements.

What is object detection?

Object detection aims to identify and localize multiple objects within images or video frames. Unlike image classification, which only classifies the broad identity of an image, object detection identifies the objects in an image/video frame and their exact positions within it.
In a nutshell, object detection solves two interdependent problems:

• Localizing (detecting) the objects on the image, by drawing the bounding boxes for the objects on the image (it is possible that there are zero objects!). A bounding box is usually defined as a tuple (x, y, h, w), where x and y are the top-left coordinates of the bounding box rectangle, and h and w are the height and width of the bounding box, respectively. 
• Classifying the identities of these images (like a person, car, or dog).

This dual capability makes object detection more complex than classification alone, requiring models that can handle multiple objects of different sizes appearing anywhere in an image.

As with classification tasks, a simple accuracy metric is not sufficient to assess model performance. We need metrics of two types. Firstly, performance metrics that gauge the trade-off between incorrectly detecting objects (false positives) and not detecting objects at all in the image when they were present (false negatives). Secondly, we also need metrics to assess how long it will take our model to perform the task in question: We will call these compute efficiency metrics. Usually, the new architectures for object detection are benchmarked on the validation partition of the COCO dataset and run on T4 NVIDIA GPU hardware.

Here are the standard metrics used in the object detection community: 

• Basic building block of performance metric: Intersection over union (IoU) is the foundational geometric measure used to decide whether a predicted bounding box is correct. It is calculated as the area of overlap between the predicted box and the ground-truth box, divided by the area of their union – producing a score between 0 (no overlap) and 1 (perfect match). A detection is counted as a true positive only if its IoU with the nearest ground-truth box exceeds a chosen threshold (e.g. 0.5). A low IoU threshold is lenient about box placement; a high one demands tight localization.
• Performance metric: Mean average precision (mAP), which evaluates detection accuracy by measuring how well predicted boxes overlap with ground truth annotations across different confidence thresholds. The most commonly cited variant, mAP@[50:95] (also written AP50:95), averages precision over IoU thresholds from 0.50 to 0.95 in steps of 0.05, which is a stringent measure that penalizes imprecise localization as much as missed detections.
  • mAP50 vs. mAP50:95: mAP50 measures detection at IoU ≥ 0.5 and scores appear higher, favoring faster models. mAP50:95 averages across IoU thresholds 0.5–0.95 – the stricter, preferred metric. For precision-critical applications (robotics, medical), it is common to optimize for mAP50:95.

• Compute efficiency metric: Frames per second (FPS), which measures inference speed, determining whether a model can process video in real-time. For standard videos, real-time is defined as >= 30 FPS (Google or original YOLO paper) or latency <= 33.3ms (1/FPS * 1000). Naturally, for such applications as self-driving cars or robotics, there are higher requirements on the FPS rate, going up as high as 60–100+ FPS.
• Compute efficiency metric: Parameter count is a quality of the model that influences its performance. There is a trade-off between the model’s accuracy and its parameter count. That’s why models are provided in different sizes of the same architecture (S, M, L, XL, etc.) to cater to various scenarios of this trade-off. This is similar to the concept of parameter count in LLMs.

There are a few popular choices of datasets to evaluate the performance of object detection models. As mentioned above, the standard choice for benchmarking object detection is the COCO dataset, containing 80 object categories across 330,000 images. Naturally, there are a lot of other datasets, specialized for certain domains, such as self-driving cars, or certain scenarios, such as the detection of objects in cluttered environments. What is important to remember is that the values of object detection metrics, IoU, and mAP depend on the dataset they were evaluated on, so mAP@[50:95]=60.1 on the COCO dataset may not be directly transferable to your custom dataset. These metrics should always be re-evaluated on your dataset to define the baseline performance of the models on it.

Object detection algorithms and architecture families

Object detection models fall into two different processing flows and two different architectural families.

Architectures

CNN-based

Examples: Faster R-CNN, Mask R-CNN, Cascade R-CNN, YOLO

CNN-based detectors rely on convolutional layers to extract local features hierarchically across the image, traditionally using predefined anchor boxes as spatial priors for localizing objects. Spatial priors are predefined assumptions about where and what size objects are likely to appear in an image, giving the model a starting point for detection rather than searching randomly.

Transformer-based

Examples: RF-DETR, RT-DETR, D-FINE

Transformer-based detectors, inspired by advances in natural language processing, instead apply global self-attention mechanisms that allow the model to reason about relationships across the entire image simultaneously.

Specifically, transformer-based detectors use learned object queries and global self-attention, where each query is trained to correspond to at most one object, unlike CNN-based detectors, which build spatial understanding locally through convolutional layers with limited receptive fields.

However, in modern architectures, there exists a fusion of the two architectures: a CNN network can use self-attention modules in its architecture, such as YOLOv12 or YOLOv13, leading to cross-architectural designs.

Processing flows

Two-stage detectors

Examples: Faster R-CNN, Mask R-CNN, Cascade R-CNN

The network makes two sequential passes, each with a distinct job:

• Stage 1: Region Proposal:
  • Scans the image and proposes ~1000–2000 candidate regions (RoIs) that might contain objects.
  • Doesn’t care about class yet, just the fact that “something interesting is here”.
  • This is the region proposal network (RPN) in classic detectors
• Stage 2: RoI classification and refinement:
  • Takes only the proposed regions from Stage 1.
  • Crops/pools features for each region.
  • Predicts the exact class and refined box coordinates for each proposal.

Single-stage detectors

Examples: YOLO series, SSD, RetinaNet, DETR

The network directly predicts class labels and bounding boxes from feature maps. It does everything in one forward pass. Usually, the following happens:

• A dense grid of anchor boxes (or points) is placed over the image.
• For each anchor, the network simultaneously predicts:
  • Whether there is an object there (objectness/class score).
  • How the box should be adjusted (box regression offsets).
• In older versions of single-stage detectors, one needed to filter overlapping bounding boxes at the end; it was done using the non-maximum suppression (NMS) algorithm. From YOLOv10 on, using NMS is a redundant step.

Furthermore, modern single-stage detectors have moved away from anchor-based designs entirely, predicting box coordinates directly from grid points and pixels, eliminating the need for dataset-specific anchor tuning altogether.

Historically, two-stage detectors offered better accuracy at the cost of speed, but modern single-stage detectors have largely closed this gap, achieving comparable or superior results while remaining significantly faster. Thus, we will focus on single-stage detectors only when evaluating the state-of-the-art models for practical applications.

Top object detection models in 2026

Two-stage pipelines (Faster R-CNN, Mask R-CNN) are no longer competitive. The current frontier is defined by single-stage NMS-free transformer architectures and models of the YOLO family. Each model below excels in a specific deployment scenario.

RF-DETR (by Roboflow) – Highest Accuracy

Real-Time Detection Transformer · ICLR 2026

The strongest real-time model available. RF-DETR uses DINOv2 to extract deeply rich, globally-aware feature representations of the input image, then uses deformable cross-attention in the detection head to efficiently query those features and predict bounding boxes without needing anchor boxes or NMS post-processing. The result is a model that’s simultaneously more accurate on complex scenes and faster at inference than the naive combination of those components would suggest. RF-DETR is the first real-time detector to break 60 mAP on MS COCO. Designed from the ground up for fine-tuning, DINOv2 pre-training on internet-scale data gives it unmatched domain adaptability across aerial imagery, medical scans, industrial inspection, and more. It comes in four sizes: Nano, Small, Medium, Large (plus XL/2XL under a PML license).

Strengths:

• Highest mAP of any real-time model.
• Exceptional domain transfer (fine-tunes fast).
• Best on occluded and complex scenes.
• Supports detection + segmentation in a single API.
• Apache 2.0, fully commercial-friendly.

Limitations:

• Heavier than YOLO on edge/mobile.
• XL/2XL models require a PML license.
• Higher GPU memory vs. YOLO variants.

License: Apache 2.0 (N/S/M/L) · PML 1.0 (XL/2XL) 

Repository: https://github.com/roboflow/rf-detr

YOLO12 (Tsinghua University) – Research / Benchmark

Attention-centric YOLO · NeurIPS 2025

YOLO12 is the first YOLO model to place attention mechanisms at the core rather than CNNs, matching CNN-based inference speeds while gaining the global context benefits of self-attention. Key innovations: Area attention (A²) divides feature maps into regions to reduce the quadratic cost of full self-attention; Residual ELAN (R-ELAN) stabilizes training of large attention blocks; FlashAttention reduces memory bottlenecks. It is deployable on NVIDIA Jetson, NVIDIA GPUs, and macOS.

A note on implementations. YOLO12 exists in two separate codebases, and the distinction matters in practice. The original authors (Tsinghua/University at Buffalo) actively maintain their own repository at sunsmarterjie/yolov12. In June 2025, they explicitly warned against using Ultralytics’ integration, stating it “is inefficient, requires more memory, and has unstable training” – issues they have fixed in their own repo. The training instability and memory criticisms often cited against YOLO12 are therefore criticisms of the Ultralytics port, not the model itself. Ultralytics’ recommendation to prefer YOLO26 over YOLO12 should be read with this context in mind: The comparison is partly against their own suboptimal implementation.

If you use YOLO12, install from the original repository rather than via pip install ultralytics.

Strengths:

• Strong accuracy at the nano scale (beats YOLO11-N by 0.9% mAP).
• Long-range context via attention mechanisms: It can take into account the entire image when detecting an object, rather than a local pixel neighborhood, as in pure CNN architectures.
• Jetson-, Android-, and macOS-deployable.
• Original repo fixes memory and training stability issues present in the Ultralytics port.
• Actively maintained by original authors with ongoing updates (turbo variant, segmentation, and classification).

Limitations:

• If using Ultralytics implementation:
  • AGPL-3.0 commercial use requires an enterprise license.
  • Training instability and high memory on large models.
• If using an open-source implementation:
  • AGPL-3.0 commercial use requires an enterprise license.
  • Claims to have stable training and inference in comparison with Ulitralytics implementation.
  • Requires installing from the original repo to avoid Ultralytics port issues, resulting in slightly more setup friction.
  • Smaller ecosystem and community support than Ultralytics-native models.

License: AGPL-3.0 (open-source) · Enterprise license via Ultralytics for commercial use 

Open-source repository: https://github.com/sunsmarterjie/yolov12

Ultralytics repository: https://github.com/ultralytics/ultralytics

YOLO26 (Ultralytics) – Best for edge / production

Edge-first unified YOLO · September 2025

Ultralytics’ flagship for 2025–2026. YOLO26 shifts focus from accuracy maximization toward deployment-oriented simplification: It removes NMS and distribution focal loss (DFL) for end-to-end inference, introduces the MuSGD optimizer for stable convergence, and adds progressive loss balancing (ProgLoss), which makes sure that the model doesn’t over-optimize one objective at the expense of others, and small-target-aware label assignment (STAL), which ensures extra attention to small objects. Five tasks are solved by this one YOLO26: detection, segmentation, pose estimation, oriented bounding boxes detection, and open-vocabulary detection and segmentation. It is explicitly designed for NVIDIA Jetson Orin/Xavier, Qualcomm Snapdragon AI, and ARM CPUs. Supports INT8 and FP16 quantization, plus ONNX, TensorRT, CoreML, and TFLite export.

Strengths:

• Best edge and mobile performance (Jetson Orin and Snapdragon).
• NMS-free leads to lower latency. 
• 43% faster CPU inference than YOLO11(N) at comparable accuracy, ideal for devices without a GPU.
• Five tasks in one architecture.
• Stable INT8/FP16 quantization.

Limitations:

• AGPL-3.0: commercial use requires an enterprise license.
• Lower peak accuracy than RF-DETR XL.

License: AGPL-3.0 (open-source) · Enterprise license via Ultralytics for commercial/industrial use.

Repository: https://github.com/ultralytics/ultralytics

Benchmark comparison

To give a comparison between the models, here are the exact benchmark values. All scores on MS COCO val2017. Latency was measured on an NVIDIA T4 GPU.

Here is a visualization of the above results alongside additional modern object detection models for a more holistic comparison:

Use-case guidance

Occluded objects: RF-DETR (M/L) is the clear choice. Its DINOv2 backbone models global context across the full image, making it significantly better than CNN-based models at finding partially hidden objects.

Small objects: RF-DETR uses multi-scale feature extraction. YOLO26 also includes STAL (small-target-aware label assignment), making it competitive for small objects on edge hardware.

Edge / mobile / Jetson: YOLO26-N or YOLO12-N. YOLO26 is the Ultralytics recommendation for Jetson Orin/Xavier, Snapdragon AI, and ARM CPUs. It has 43% faster CPU inference than YOLO11n at comparable accuracy.

Custom domain / fine-tuning: RF-DETR by a significant margin. DINOv2 pre-training means it adapts to new domains (medical, aerial, and industrial) faster and with less data than any other model here.

Licensing Summary

Quick-start code

RF-DETR

# Install
pip install rfdetr

# Inference
from rfdetr import RFDETRBase
model = RFDETRBase()
detections = model.predict("image.jpg")

# Fine-tune on your dataset
model.train(dataset_dir="./my_dataset", epochs=50, batch_size=4)

YOLO26 / YOLO12 (via Ultralytics)

# Install
pip install ultralytics

# Inference — YOLO26
from ultralytics import YOLO
model = YOLO("yolo26n.pt")          # or yolo26s/m/l/x
results = model.predict("image.jpg")

# Inference — YOLO12
model = YOLO("yolo12n.pt")
results = model.predict("image.jpg")

# Export for edge (TensorRT / CoreML / ONNX)
model.export(format="engine")       # TensorRT for Jetson
model.export(format="coreml")       # Apple Silicon / iOS
model.export(format="tflite")       # Android / ARM

YOLO12 (use original open-source repo – not the Ultralytics integration)

# Install from the original authors' repo
conda create -n yolov12 python=3.11
conda activate yolov12
git clone https://github.com/sunsmarterjie/yolov12 && cd yolov12
pip install -r requirements.txt
pip install -e .

# Inference
from ultralytics import YOLO
model = YOLO("yolov12n.pt")         # or s/m/l/x
results = model("path/to/image.jpg")
results[0].show()

# Export for edge
model.export(format="engine", half=True)   # TensorRT FP16
model.export(format="onnx")                # ONNX for broad compatibility

Transfer learning and fine-tuning

RF-DETR – recommended for domain shift. Thanks to a DINOv2 backbone that is pre-trained on internet-scale data, fine-tuning requires less labeled data and converges faster. Use the rfdetr package with a COCO pre-trained checkpoint. Roboflow also offers a hosted fine-tuning UI.

YOLO26 / YOLO12 – easiest pipeline. Ultralytics’ training API is the most mature fine-tuning ecosystem. It supports YOLO-format and COCO-format datasets and has good documentation and an active community.

# Fine-tuning YOLO26 on a custom dataset (YOLO format)
from ultralytics import YOLO

model = YOLO("yolo26m.pt")          # start from pretrained weights
model.train(
    data="custom_dataset.yaml",     # path to your dataset config
    epochs=100,
    imgsz=640,
    batch=16,
    device=0,                       # GPU index; "cpu" for CPU
)
metrics = model.val()               # evaluate on validation set

Summary: Choosing the right model for your project

Selecting an object detection model requires matching your specific requirements against each model’s strengths. The decision framework below maps common scenarios to optimal model choices.

Get started with PyCharm today

Selecting an object detection architecture in 2026 is a strategic decision dictated by the specific requirements of the application and the available computational budget. Whether prioritizing the record-breaking accuracy of RF-DETR for complex scenes or the unmatched efficiency of the YOLO family for edge deployment, the choice must balance mAP requirements against real-time latency constraints.

The landscape of computer vision is rapidly shifting toward zero-shot detection frameworks that recognize novel objects without task-specific supervision. As foundation models increasingly integrate sophisticated image embedders like CLIP or DINOv2 into detection pipelines, the boundaries of high-precision detection on resource-constrained hardware will continue to expand. While transformer-based architectures are developing quickly, the YOLO family’s established ecosystem ensures it remains a cornerstone for real-time production environments.

To achieve the best results for your specific use case, we encourage you to experiment with the models and code samples provided in this guide. To that end, PyCharm provides the perfect ecosystem for experimentation with various open-source models via Code -> Insert HF Model interface. If you’d like to try this yourself, PyCharm Pro comes with a 30-day trial.

For a hands-on starting point, this tutorial shows how to build a live object detection app using TensorFlow and PyCharm Jupyter notebooks, then deploy it on a robot – covering everything from single-frame inference to a live web dashboard with annotated detections. Moreover, stay tuned for the next tutorial post, where we will discuss all three object detection models in action.

Overview of explicit lazy imports

PEP 810 introduces an explicit syntax for lazy imports, allowing you to defer the loading and execution of modules until their attributes are actually accessed, unlike standard eager imports that execute immediately. This feature aims to significantly reduce startup latency and memory consumption. Explicitly marking modules as `lazy` can deliver substantial improvements in initial responsiveness and baseline resource usage in large-scale applications and command-line tools.

Because the implementation approach uses proxy objects within the module’s namespace instead of modifying Python’s fundamental dictionary structures, it preserves critical interpreter optimizations. 

This mechanism defers both the finding and the loading of the module to maximize efficiency, especially in environments with high-latency filesystems. To manage potential side effects and ensure backward compatibility, the proposal includes global control flags and a transitional variable for progressive adoption across different Python versions.

In short, Python 3.15 will let you optimize application performance by significantly reducing startup latency and memory consumption, as the loading and execution of modules are deferred until their attributes are actually accessed.

Trying them out in Python 3.15.0b1

At the time this is being written, Python 3.15.0b1 is already out, so we can give this new feature a try. You can build it from source at the CPython GitHub repo, but since getting Python 3.15.0b1 is easy when using `uv` or `pyenv`, we will do that instead.

Make sure you have the latest version of `uv` or `pyenv`, and then download Python 3.15.0b1 via either of the following commands:

• `uv python install 3.15.0b1`
• `pyenv install 3.15.0b1`

After that, select the new interpreter in your project in PyCharm.

Now you will need to reinstall the dependencies for your project. You may have to build some of the libraries from source, as most of the libraries will not have a Python 3.15 wheel for download.

Profiling against normal imports

It is a common joke that the first thing data scientists will do is type `import pandas as pd` and `import numpy as np`, even if they are not actually going to use them. Let’s assume this is the case, and you received a script like this from your colleague:

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

def main():
    print("Initializing example data science project...")
    
    # Generate some dummy data
    data = {
        'x': np.linspace(0, 10, 100),
        'y': np.sin(np.linspace(0, 10, 100)) + np.random.normal(0, 0.1, 100)
    }
    
    # Plotting
    plt.figure(figsize=(10, 6))
    plt.plot(data['x'], data['y'], label='Sine Wave with Noise')
    plt.title('Sample Visualization')
    plt.xlabel('X-axis')
    plt.ylabel('Y-axis')
    plt.legend()
    
    # Save the plot instead of showing it (since this is non-interactive)
    plt.savefig('sine_wave.png')
    print("Project executed successfully. Plot saved as sine_wave.png.")

if __name__ == "__main__":
    main()

As you see, PyCharm highlights the unused pandas import for you, so removing it would be straightforward. However, for our experiment here, we’ll keep it.

To get a better visualization of import profiles, install a tool from PyPI called tuna.

You can profile your script by setting a custom run script with this script text:

python -X importtime main.py 2> import_log.txt; tuna import_log.txt

When you use it, a new browser window will pop up with the import graph. 

As you see, importing pandas accounts for half of the time it takes to load all the modules, and we never use it!

Now let’s add `lazy` to all the imports.

Don’t worry about the syntax highlighting. PyCharm just doesn’t recognize it yet since `lazy` is a new keyword that has not been officially released.

Let’s profile the script again.

Now we see the pandas import is gone, and loading everything takes way less time.

So, if you have a script that imports a lot of large libraries, and some of them are only used in certain conditions (e.g. in if-else clauses), lazy import can save time by loading modules only when they are first used.

Checking the inner workings with lazy imports

Let’s see how lazy imports are handled internally.

When a module is imported “lazily”, meaning `__lazy_import__` is called instead of `__import__`, a `types.LazyImportType` proxy object will be created. The module name will then be listed in `sys.lazy_modules` instead of `sys.modules`. (See the Lazy import mechanism section in PEP 810.)

When a lazy object is used, it needs to be reified. CPython will try resolving the import at that point and replacing the proxy object with the actual module itself. In this process, `__import__` is called to resolve the import. At the same time, the module is removed from `sys.lazy_modules`.

If there’s an error during reification, AKA importing the module, the lazy object is not reified or replaced. The next time the lazy module is used, the import will try again. The exception raised during reification will also show both where the lazy import was defined and where it was accessed. (See the Reification section in PEP 810.)

To experiment with it ourselves, let’s add some breakpoints with `pdb` and check what’s happening in the code:

import pdb
pdb.set_trace()

lazy import pandas as pd
lazy import matplotlib.pyplot as plt
lazy import numpy as np

pdb.set_trace()
…

And

    # Generate some dummy data
    data = {
        'x': np.linspace(0, 10, 100),
        'y': np.sin(np.linspace(0, 10, 100)) + np.random.normal(0, 0.1, 100)
    }

    pdb.set_trace()
    
    # Plotting
    plt.figure(figsize=(10, 6))
    pdb.set_trace()
…

Now run the script in the console:

python main.py

Note that PyCharm 2026.1 does not yet support Python 3.15, so using the Run or Debug button to run a script using lazy import may result in unexpected behavior.

When it hits the first line of ` pdb.set_trace()` at the top, there should not be any module loaded in. Let’s check:

(Pdb) import sys
(Pdb) sys.lazy_modules

As expected, none of our libraries – pandas, numpy, and matplotlib – are listed.

Now, let’s continue running the program and let it stop at the next breakpoint. In the console, type `continue` and once it stops, we can check by typing `sys.lazy_modules` again:

Here, we see that all of our modules are in `lazy_modules`. Let’s check whether pandas is in `sys.modules`:

(Pdb) 'pandas' in sys.modules

Nope, it’s not. You can try with numpy and matplotlib, and you will see that neither of those is in `sys.module`.

Now let’s type `continue` again and reach the next breakpoint, which occurs after numpy is used. Check `sys.lazy_module` again, and you’ll see that numpy is no longer on the list. When we check whether it is in `sys.module`, we get `True` this time.

However, pandas and matplotlib are still not in `sys.modules`.

When you check the next breakpoint, you’ll see that matplotlib is similarly removed from `sys.lazy_modules` and added to `sys.modules` after it is used.

Trying it yourself with PyCharm

Download the latest version of PyCharm to experiment with Python 3.15.0b1 and experience firsthand how explicit lazy imports can optimize your application’s performance by significantly reducing startup latency and memory consumption.

We’re collaborating with LinkedIn to make it easier for you to showcase your expertise with JetBrains IDEs on the world’s largest professional network. You can now connect your IDE to LinkedIn and let your real tool usage speak for itself.

Connect your IDE

IntelliJ IDEA, PyCharm, WebStorm, PhpStorm, Rider, GoLand, CLion, RustRover, and RubyMine are already supported via a free plugin, while support for DataGrip is coming soon.

In this blog post, we’ll explain what LinkedIn connected apps are, what they mean for your profile, and how to get started.

What this is about

Building on early collaboration with Descript, Duolingo, Lovable, Relay.app, and Replit, LinkedIn is expanding the range of apps you can feature on your LinkedIn profile, turning real-world product usage into a credible, visible signal of practical tool experience. We’re glad to join forces with them to bring this to JetBrains IDE users.

“We’re building new ways for members to show real, credible proof of what they’re capable of, right on their LinkedIn profile. And for the brands behind these tools, there’s no better endorsement than a customer who’s actively using and loving your product.”
– Dan Shapero, CEO of LinkedIn

Connected apps let you link the tools you use in your daily work directly to your LinkedIn profile, where they appear prominently, helping you stand out to your professional network. Once connected, each app generates a simple statement based on how you actually use it. Unlike manually added skills, this is based on real usage and updates automatically as your experience evolves.

How to get started

Open your JetBrains IDE, go to Settings | Plugins, search for the LinkedIn Connected Apps plugin under the Marketplace tab, and install it. 

Once installed, the plugin starts collecting data locally about how you use your IDE. Depending on your usage history, you may receive an initial statement right away, which will then be updated once the plugin has collected enough data to better reflect your real IDE expertise.

Your IDE usage data stays on your machine. When you are ready, you can connect your LinkedIn account and share your IDE expertise badge there. If you keep the plugin installed, your badge will update automatically as your IDE usage evolves.

The plugin is free for all JetBrains IDE users.

What’s included in this release

This is the first version of the integration, delivered as a standalone plugin rather than being built directly into the IDE. It covers IntelliJ IDEA, PyCharm, WebStorm, GoLand, PhpStorm, Rider, CLion, RustRover, and RubyMine; DataGrip is not yet supported. 

Usage is detected within the IDE itself, so if you use AI features via an external tool or terminal, those won’t be reflected yet.

How your IDE expertise is determined

The model is intentionally simple for now. It is designed to represent your practical use of JetBrains tools, not to rank developers or certify skill levels. Our goal was to provide a solid starting point, but we know there’s more to capture about how developers work with their IDEs.

Statements map to different levels of experience and are generated based on how you interact with your IDE – from writing and editing code using basic features to working with debugging tools, version control, and AI-assisted workflows. For more information, see our documentation.

What’s coming next

We’re already working on the next version, planned for later this year. We’ll focus on improving how IDE usage, including AI feature usage, is represented, expanding support to DataGrip, and making the overall experience feel more integrated.

FAQ

Which JetBrains IDEs are supported?

IntelliJ IDEA, PyCharm, WebStorm, GoLand, PhpStorm, Rider, CLion, RustRover, and RubyMine. Support for DataGrip is coming soon.

Why isn’t DataGrip supported yet?

DataGrip is designed for working with databases and includes workflows that differ from our other IDEs. We plan to support it soon.

Can I connect multiple IDEs?

Yes, if you use multiple supported JetBrains IDEs, you can connect each of them. You’ll get a badge for all connected IDEs.

Note: If you use multiple JetBrains accounts across different IDE instances but link them all to the same LinkedIn profile, IDE usage statements from each account will be displayed on that LinkedIn profile.

Do I need to keep the plugin installed after connecting? 

You can share your IDE usage statement once and then remove the plugin, but note that it must remain installed if you want to track your ongoing progress and have any changes reflected on LinkedIn.

Is this feature free?

Yes, it’s available to all JetBrains IDE users at no cost.

Is this a certification?

Connected apps reflect real IDE usage and are designed to showcase applied experience, not to act as a formal certification or skill ranking. Certifications, degrees, and licenses remain important markers of professional achievement. Connected apps on LinkedIn add a different kind of signal: partner-validated tool usage that reflects practical work and can update over time.

What data is shared with LinkedIn and JetBrains?

Only the information required to represent your connected account and IDE usage statement. 

Will this help me get hired?

Having connected apps on your LinkedIn profile is extra proof of your practical experience with leading tools. While connected apps make your expertise visible, they are just one part of your profile. Think of it as a way to let your tooling speak for itself.

Give it a try and let us know what you think in the comments below. We’re continuing to develop this integration, and your feedback will help shape what comes next.

Whether you’re building chatbots, training computer vision models, or analyzing business data, choosing the right AI framework can make or break your project. Python has become the dominant language for AI and machine learning development, and the ecosystem of frameworks supporting this work has matured significantly.

The right framework choice depends on what you’re building. A production recommendation system has different requirements than a research prototype. A chatbot powered by large language models (LLMs) needs different tools than a fraud detection system analyzing tabular data.

Let’s explore seven essential frameworks and where each excels so you can find the best AI framework for your specific project.

What is an AI framework?

AI frameworks are pre-built libraries and tools that handle the complex mathematics, data structures, and computational operations underlying AI and machine learning models. Rather than implementing neural networks or gradient descent from scratch, AI frameworks provide abstractions that let you focus on model architecture, data preparation, and business logic.

These frameworks generally fall into three categories:

• Deep learning frameworks like TensorFlow, PyTorch, and Keras specialize in neural networks and GPU acceleration for tasks involving images, text, and audio.
• Classical and tabular machine learning frameworks like scikit-learn and XGBoost focus on statistical and tree-based models for structured data, powering many real-world AI systems, including forecasting, risk-scoring, and decision-automation solutions.
• LLM and AI agent frameworks like LangChain and Hugging Face provide tools for building applications powered by large language models.

Why do AI frameworks matter? 

AI frameworks dramatically accelerate your development by providing tested, optimized implementations of complex algorithms. They offer strong community support with extensive documentation, tutorials, and troubleshooting resources. They provide production-ready tooling for deployment, monitoring, and scaling. They’re optimized for specific hardware like GPUs and TPUs, delivering performance that would be difficult to achieve with custom implementations.

Open-source vs. commercial AI frameworks

Open-source AI frameworks are the dominant model in AI development today. And they offer compelling advantages, from community-driven innovation for rapid feature development and bug fixes to transparency that enables auditing and algorithm customization. There’s also no vendor lock-in or licensing fees, making them cost-effective for both experimentation and production deployment.

Commercial AI platforms also exist, with AWS SageMaker, Google Vertex AI, and Azure Machine Learning among the prominent examples. However, these platforms often use open-source frameworks underneath rather than competing with them directly. They provide managed infrastructure, automated workflows, and enterprise features on top of tools like TensorFlow and PyTorch.

If you’re thinking open source means they’re unsupported, think again. All seven frameworks below have robust ecosystems, and many are backed by major tech companies. Google supports TensorFlow, Meta backs PyTorch, and organizations like Microsoft contribute significantly to various projects in the ecosystem.

Top Python AI frameworks

These seven frameworks represent the essential toolkit for Python AI development in 2026. Each performs strongly in specific domains, and many developers use multiple frameworks depending on project requirements.

TensorFlow

TensorFlow is an open-source deep learning framework developed by Google for building and deploying machine learning models at enterprise scale. With a 37% market share in data science and machine learning and adoption by 25,000 companies globally, TensorFlow has proven itself in high-stakes production environments.

The framework evolved significantly from TensorFlow 1.x to 2.x, with Keras integration making it far more accessible while maintaining its enterprise-grade capabilities. If you’re building large-scale image recognition systems or natural language processing pipelines, or you need to deploy across web, mobile, and edge devices through TensorFlow Lite and TensorFlow.js, TensorFlow can help.

If you’re just getting started with TensorFlow, follow our step-by-step tutorial on how to train your first TensorFlow model using PyCharm.

Advantages of TensorFlow

• Enterprise-grade scalability: Built for production from day one, TensorFlow handles massive datasets and distributed training across multiple GPUs and TPUs seamlessly. You can scale from experimentation to serving millions of predictions without switching tools.
• Comprehensive deployment ecosystem: TensorFlow Serving handles model deployment, TensorFlow Lite optimizes for mobile and edge devices, and TensorFlow.js brings models to browsers. This complete deployment story reduces friction when moving from development to production.
• TPU optimization: Native support for Google’s Tensor Processing Units delivers superior performance for large-scale training workloads, offering significantly better performance per watt than traditional hardware.
• Strong industry adoption: Companies like Airbnb, Twitter, and Intel rely on TensorFlow for critical applications, giving you confidence in its production readiness and long-term viability.

Disadvantages of TensorFlow

• Steeper learning curve: Despite Keras integration, TensorFlow’s complexity can overwhelm beginners, especially when you move beyond high-level APIs to custom implementations.
• Verbose syntax for custom models: Building custom training loops or novel architectures requires significantly more code compared with PyTorch’s more Pythonic approach.
• Debugging challenges: Static graph optimization, while beneficial for performance, can make runtime errors harder to trace than in frameworks with dynamic computation graphs.

scikit-learn

scikit-learn is an open-source Python library for classical machine learning, providing simple and efficient tools for classification, regression, clustering, and dimensionality reduction. With adoption by over 16,000 companies worldwide, it’s your essential first stop for structured and tabular data before considering deep learning approaches.

The framework supports a wide range of supervised and unsupervised learning on structured business data, along with feature engineering and data preprocessing pipelines. Companies like J.P. Morgan use scikit-learn extensively for classification tasks and predictive analytics in financial decision-making.

Advantages of scikit-learn

• Beginner-friendly API: Consistent, intuitive syntax across all algorithms makes learning and switching between models effortless. The fit/predict pattern works the same whether you’re using linear regression or random forests.
• Comprehensive algorithm library: Its library covers virtually every classical ML algorithm – regression, classification, clustering, dimensionality reduction – with well-tested implementations ready for your projects.
• Excellent for tabular data: On structured data, traditional algorithms often outperform deep learning, and scikit-learn gives you the tools to maximize this advantage.
• Fast prototyping: Its simple syntax means you can build and test models in minutes, not hours, making it ideal for rapid experimentation.
• Seamless integration: scikit-learn works perfectly with NumPy, pandas, and Matplotlib, fitting naturally into your data science workflows.

Disadvantages of scikit-learn

• No deep learning support: scikit-learn is not designed for neural networks – you’ll need to switch to TensorFlow or PyTorch for complex deep learning architectures.
• Limited GPU acceleration: The framework is CPU-bound and struggles with very large datasets where GPU-accelerated frameworks perform better.
• Not suited for unstructured data: Images, text, and audio require deep learning frameworks that can handle high-dimensional, unstructured inputs.

PyTorch

PyTorch is an open-source deep learning framework developed by Meta that prioritizes flexibility and a natural Python coding experience. It’s used in approximately 85% of deep learning research papers and has a 55% adoption rate in the research community. From its academic roots, PyTorch has evolved into a production-ready powerhouse.

The framework excels at cutting-edge research and experimentation with novel architectures. It supports natural language processing and generative AI models such as GPT, Llama, and Stable Diffusion, and enables computer vision research with custom model development. Its Pythonic philosophy makes it feel natural if you’re already comfortable with Python, reducing cognitive load and accelerating your development.

Advantages of PyTorch

• Dynamic computation graphs: The define-by-run approach allows runtime model modifications, making debugging and experimentation intuitive. You can use standard Python control flow and debugging tools you already know.
• Pythonic and readable: PyTorch code feels like native Python, not a separate language. This flattens your learning curve and makes code more maintainable.
• Research-first innovation: Latest techniques and models appear in PyTorch first, driven by its dominance in academic research.
• Strong ecosystem: Hugging Face Transformers, PyTorch Lightning, and extensive community packages provide specialized tools for virtually any task you’ll encounter.

Disadvantages of PyTorch

• Deployment complexity: While TorchServe has improved the situation, PyTorch historically has had weaker production tooling compared to TensorFlow’s mature deployment ecosystem.
• Manual training loops: Greater control means more boilerplate code for standard training patterns, though libraries like PyTorch Lightning address this.

Keras

Keras is a high-level deep learning API designed for fast experimentation with neural networks. With over 60,000 GitHub stars and integration as TensorFlow’s default interface, Keras has become synonymous with rapid prototyping and ease of use. The release of Keras 3.0 changed the game by adding multi-backend support for TensorFlow, JAX, and PyTorch.

The framework is ideal for rapidly prototyping neural network architectures, working on educational projects to learn deep learning fundamentals, or tackling deep learning tasks that don’t require low-level customization.

Advantages of Keras

• Simplest API in deep learning: You can build sophisticated models in just a few lines of code with the Sequential or Functional API, offering the lowest barrier to entry in deep learning.
• Multi-backend flexibility: Keras 3.0 runs on TensorFlow, JAX, or PyTorch – write once, run anywhere. This future-proofs your code and lets you switch backends as your needs change.
• Built-in best practices: The API guides you toward sound model architecture decisions and incorporates best practices by default.
• Fast experimentation: You can iterate quickly without wrestling with framework complexity, focusing on model design rather than implementation details.

Disadvantages of Keras

• Limited low-level control: The abstraction layer sacrifices fine-grained control needed for cutting-edge research or novel architectures.
• Performance overhead: The additional abstraction can introduce latency compared to native framework calls, though this is often negligible for most applications.
• Less suitable for custom architectures: Highly novel model designs may require you to drop down to the underlying framework.

LangChain

LangChain is an open-source framework that helps you build applications powered by large language models, providing core components for prompt management, chains, memory, and agent orchestration. It acts as an abstraction layer to easily connect LLMs to external data sources and computational tools. With over 120,000 GitHub stars, the framework has become essential infrastructure for the AI agent revolution.

LangChain is most commonly used for building conversational AI and chatbots with memory and context, retrieval-augmented generation (RAG) systems for enterprise knowledge bases, and multi-agent systems with autonomous workflows.

If you want to go beyond the basics, read our LangChain Python Tutorial: A Complete Guide for 2026. It takes a deeper look at what LangChain offers and walks through real-world use cases for building AI agents in Python.

Advantages of LangChain

• Comprehensive LLM orchestration: Handles everything from prompt management to chains, memory, and tool use, giving you a complete infrastructure for LLM applications in one package.
• Provider-agnostic: Works seamlessly with OpenAI, Anthropic, Hugging Face, and local models, letting you avoid vendor lock-in and switch providers as your needs change.
• Rich agent capabilities: LangGraph enables complex, stateful workflows with human-in-the-loop patterns, supporting sophisticated agentic behaviors.
• Production-ready tooling: LangSmith provides monitoring, debugging, and tracing specifically designed for LLM applications, addressing the unique challenges you’ll face in production.

Disadvantages of LangChain

• Learning curve for abstractions: LangChain Expression Language (LCEL) and framework-specific concepts take time to master, especially if you’re new to LLM orchestration.
• Abstraction overhead: Additional layers between you and LLM APIs can sometimes obscure what’s happening, making debugging more challenging.
• Fast-moving target: Frequent updates mean your code can become outdated quickly, requiring ongoing maintenance to stay current.

Hugging Face

Hugging Face is an open-source platform and library ecosystem for natural language processing and machine learning, with over one million models and 250,000 datasets to power your next project. It’s become a central hub for the AI community, with its Transformers library earning 150,000+ GitHub stars.

The platform is particularly effective at accessing and fine-tuning pre-trained transformer models like BERT, GPT, and Llama, building NLP applications without training models from scratch, and sharing and deploying custom models to the community.

For a practical example, read A Practical Guide to Fine-Tuning and Deploying GPT Models Using Hugging Face Transformers. It walks through using a pre-trained GPT model, fine-tuning it on custom data, and deploying the result with FastAPI.

Advantages of Hugging Face

• Massive model repository: With hundreds of thousands of pre-trained models available, you rarely need to train from scratch. Models for virtually every task and language are ready for you to use.
• Transformers library dominance: This is the de facto standard for NLP, computer vision, and multimodal models, with support for the latest architectures as soon as they’re published.
• Framework interoperability: Models work with PyTorch, TensorFlow, and JAX, giving you maximum flexibility in your development workflow.
• Inference infrastructure: Hosted inference APIs and Spaces make deployment straightforward without managing your own infrastructure.

Disadvantages of Hugging Face

• Dependency complexity: The large dependency tree can lead to version conflicts and package management challenges, especially in complex environments.
• Model quality variance: Community-contributed models vary in quality and may not be production-ready without thorough vetting and testing on your part.
• Platform dependency: Heavy reliance on Hugging Face Hub creates some platform lock-in, though you can download models and host them independently.

XGBoost

XGBoost is an optimized gradient boosting library designed for speed and performance on structured data. The algorithm continues to dominate machine learning competitions alongside other gradient-boosted decision tree libraries, earning its reputation through battle-tested performance on real-world problems.

You can use the framework for predictive modeling on structured business data, including sales forecasting, risk assessment, and feature importance analysis for model interpretability. Its gradient-boosting approach achieves outstanding precision on structured data, powering reliable insights for business applications.

Advantages of XGBoost

• Superior accuracy on tabular data: XGBoost consistently outperforms deep learning on structured datasets, making it your default choice for business analytics and forecasting.
• Built-in regularization: L1 and L2 regularization prevents overfitting better than basic gradient boosting, producing more robust models for your production systems.
• Efficient computation: Handles large datasets efficiently with parallel processing and intelligent tree pruning, making it practical for production use.
• Missing value handling: Automatically learns optimal strategies for missing data, reducing your preprocessing burden.
• Feature importance scores: Built-in interpretability helps you understand model decisions, crucial for business applications and regulatory compliance.

Disadvantages of XGBoost

• Not suitable for unstructured data: Images, text, and audio require deep learning approaches. XGBoost is designed specifically for tabular data.
• Hyperparameter complexity: There are many parameters to tune for optimal performance, though tools like Optuna can automate this process for you.
• Limited interpretability compared with simple models: While more explainable than deep neural networks, XGBoost’s ensemble structure is harder to interpret than linear or rule-based models, even with feature importance and SHAP analysis.

How to choose an AI framework

Selecting the best AI framework depends on your specific project characteristics, but in practice, the choice is rarely binary. Many successful teams use multiple frameworks together. A common and effective pattern is to use scikit-learn for preprocessing and feature engineering, PyTorch for research and model development, TensorFlow for production deployment, and LangChain for LLM-powered features.

Your decision will likely come down to data type, team expertise, and where your model needs to run. Use this table as a starting point:

If your choice comes down to PyTorch or TensorFlow, read our dedicated PyTorch vs. TensorFlow: Choosing the Right Framework in 2026 guide, where we compare learning curves, deployment options, and use cases to help you choose the right deep learning framework.

This rapid shift means that Python developers building autonomous systems are increasingly relying on agentic frameworks to manage reasoning, memory, tools, and collaboration among multiple agents.

You’ve probably already heard of some of the most popular frameworks. LangChain and AutoGen have risen to prominence, but there are dozens more, many of them open-source and only one to two years old. With so many frameworks promising different agentic capabilities, the real challenge is knowing which ones are best suited for the kind of application you want to build.

Let’s take a closer look at some of the most important agentic frameworks on the market in 2026, comparing what each does best and rating them based on our key comparison criteria to help you discover which is best for your projects.

What are AI agents?

An AI agent is a piece of software capable of autonomously reasoning, setting goals, and performing tasks on behalf of a user or another system. As the name suggests, AI agents have a level of agency to learn, adapt, and make decisions independently. This means they can improve their behavior and, over time, choose their own actions to achieve specific goals or outcomes.

AI agents work by following a perceive, reason, act, reflect (PRAR) cycle, which allows them to:

• Perceive: Observe the environment, including user input, system state, tools, and memory, to understand the current context and constraints of the task.
• Reason: Plan, make decisions, and select actions using a large language model (LLM) or hybrid logic.
• Act: Execute actions like calling tools, updating memory, or triggering workflows.
• Reflect: Evaluate the outcome of previous actions and adjust future decisions, plans, or prompts to improve results.

AI agents rely on the natural language processing capabilities of large language models, but unlike traditional LLMs and AI chatbots, they don’t require continuous user input to perform tasks. Agents are proactive, working autonomously to achieve a goal based on a specified set of rules and parameters.

What is an agentic framework?

An agentic framework provides the infrastructure needed to build, run, and control AI agents at scale. Most modern frameworks offer three core capabilities:

• Orchestration: Controls how agents are sequenced, coordinated, or allowed to collaborate.
• Tools: Define how agents interact with external systems like APIs or databases.
• Memory: Sets out how agents retain and retrieve information across steps or sessions.

While it’s possible to build an agent without a framework, they’re vital in ensuring agents are reliable, scalable, and safe.

Agentic frameworks help turn experimental agent builds into maintainable software by facilitating:

• Multi-agent coordination: When multiple agents communicate to plan, work together, and specialize in different areas of a task.
• Human-in-the-loop (HITL) checkpoints: Intentional pause points where a human can review what an agent is about to do.
• Observability, control, and reproducibility: The ability to see what an agent is doing, guide agent behavior, or re-run an agent and receive the same results.

Core orchestration paradigms

Before comparing individual frameworks, it’s important to understand how they operate. Let’s look at the three most commonly used orchestration models in 2026.

Graph-based orchestration

Graph-based orchestration provides maximum control by organizing agents and tools as nodes in a directed graph. Instead of letting an agent freely decide what to do next, the flow that agents are allowed to follow is clearly defined.

Strengths

• More deterministic control: Predictable behavior is critical for production systems that require reliable results.
• Easier debugging: Pinpoint exactly which node failed thanks to clear checkpoints and boundaries.
• Production-grade reliability: This approach is ideal for customer-facing applications, enterprise systems, or regulated environments.

Limitations

• More upfront design: The workflow must be defined in advance, which slows initial development.
• Less “emergent” behavior: Agents are constrained by the graph, leaving less room for experimentation and creativity.

Role-based orchestration

Role-based orchestration is most effective when simplicity is a priority. Agents are assigned specific roles, such as “Planner”, “Researcher”, or “Builder”, and collaborate by sending messages to one another.

Strengths

• Intuitive mental model: This type of operation is easy to understand because it effectively mirrors how human teams work.
• Rapid prototyping: Minimal setup is required, allowing more time to explore outcomes.

Limitations

• Harder-to-constrain behavior: Because agents have the freedom to decide what to do next, it’s difficult to enforce strict execution paths.
• Limited determinism: The same input can yield different outcomes, making it tricky to reproduce results and achieve consistency.

Chain-based orchestration

Chain-based orchestration, also known as adaptive orchestration, arguably offers the greatest flexibility. Agents in this model operate in dynamic chains or loops, deciding the next step autonomously.

Strengths

• Flexible workflows: Agents are not constrained to a pre-defined path and can freely explore different strategies.
• Suitability for creative tasks: This approach is ideal for research, discovery, and experimentation, as agents can iteratively explore ideas, pivot strategies, and adapt their approach.

Limitations

• Less predictability: Testing and debugging are more challenging because execution paths are harder to reproduce and trace.
• More difficult governance at scale: This unpredictability grows as tasks become more complex.

Best agentic frameworks for your projects

Now that we’re familiar with the key orchestration paradigms of agentic frameworks, it’s time to compare some of the most popular frameworks on the market in 2026. Below, we evaluate each framework’s performance against our key comparison criteria:

• Primary orchestration model.
• Multi-agent support.
• Memory capabilities.
• Human-in-the-loop (HITL) support.
• Best-fit applications.

Let’s take a closer look at the strengths and weaknesses of each framework, along with the applications they’re most suited to.

LangChain

• Core design: Chain-based orchestration.
• Philosophy: Developer velocity and flexibility.

Launched in 2022, LangChain is one of the most widely adopted frameworks due to its broad ecosystem of integrations. It serves as an accessible interface for nearly any LLM and is an ideal starting point for enthusiasts or startups looking to explore agentic AI. While not strictly “agent-first”, it provides the building blocks for agentic behavior.

LangChain provides less control than other frameworks, but it’s still a fantastic entry point into agentic systems, especially for projects where speed and creativity take precedence over enforcing strict workflows.

Strengths

• Huge ecosystem.
• Easy tool integration.
• Rapid prototyping.

Limitations

• Less control than graph-based systems.
• Agent logic that can be difficult to understand as it grows in complexity.

Best applications

• Prototyping of agentic features.
• Tool-augmented chatbots.
• LLM-powered backend services.

If you want to go beyond the basics, read our LangChain Python Tutorial: A Complete Guide for 2026. It takes a deeper look at what LangChain offers and walks through real-world use cases for building AI agents in Python.

LangGraph

• Core design: Graph-based orchestration.
• Philosophy: Explicit control over agent behavior.

LangGraph has emerged as the leading standard for production-grade agent systems. Built on top of LangChain, it replaces implicit chains with explicit graphs, providing strict control over workflows and excellent HITL support via interrupts.

While the graph structure itself can actually make debugging easier by clearly mapping how agents and tools interact, LangGraph does come with a learning curve. Much of this complexity comes from designing the graph and managing explicit state between nodes. Once you understand these concepts, the framework becomes a powerful option for building predictable and controllable agent systems.

Strengths

• Deterministic workflows.
• Native state management.
• Excellent HITL support via interrupts.
• Suitability for regulated or mission-critical systems.

Limitations

• Higher upfront design effort.
• Steeper learning curve due to explicit graph and state management.
• Reduced flexibility for open-ended tasks.

Best applications

• Autonomous customer support systems.
• AI-driven DevOps workflows.
• Multi-step decision engines.

LlamaIndex

• Core design: Retrieval-centric orchestration.
• Philosophy: Data-first agents.

LlamaIndex is a Python framework designed to help AI systems understand, store, and retrieve information from large amounts of documents and data.

Rather than starting with agents and adding data later, LlamaIndex takes the opposite approach – it starts with data and then builds agent behavior around it. This is why it is often described as data-first or retrieval-centric.

Because it operates in this way, LlamaIndex excels at indexing, memory, and retrieval, making it ideal for building agents whose intelligence depends on accessing the right information rather than executing complex actions.

Strengths

• Advanced document indexing.
• Strong long-term memory patterns.

Limitations

• Limited suitability for complex, action-heavy orchestration.
• Limited support for multi-agent orchestration.

Best applications

• Research assistants.
• Knowledge base agents.
• Enterprise document intelligence.

Haystack

• Core design: Modular pipeline orchestration.
• Philosophy: Context engineering and production-ready AI systems.

Haystack is an open-source AI orchestration framework created by deepset for building production-ready AI agents, retrieval-augmented generation (RAG) systems, and multimodal applications.

Instead of focusing purely on agent behavior, Haystack structures applications as explicit pipelines composed of retrievers, routers, memory layers, tools, evaluators, and generators. This modular architecture gives you control over how information flows through a system, allowing each component to be tested and improved independently.

Haystack is particularly strong in applications where the quality of retrieved information determines the quality of the model’s output. Its design also makes it well-suited for enterprise environments that require transparency and reliability in production systems.

Strengths 

• Highly modular pipeline architecture.
• Excellent support for RAG and document processing.

• Strong ecosystem, particularly in search and RAG-focused enterprise use cases.
• Flexible integrations with models and vector databases.

Limitations 

• More infrastructure and setup than lightweight frameworks.
• Less focus on emergent multi-agent collaboration.

Best applications

• Retrieval-augmented generation (RAG) systems.
• Enterprise document intelligence.
• Data-heavy AI applications.
• Production AI pipelines that require strong context control.

AutoGen

• Core design: Role-based multi-agent collaboration.
• Philosophy: Conversation-driven autonomy.

AutoGen, an open-source Microsoft framework, popularized the idea of agents collaborating through structured conversation, organizing systems as teams of agents, each with its own specific role. Unlike in other frameworks, there’s no central controller enforcing a strict execution path – the collaboration itself drives progress.

This approach makes AutoGen ideal for exploratory, creative, and research-driven multi-agent systems, at the cost of predictability, HITL, and strict execution control.

Strengths 

• Natural multi-agent interaction. 
• Minimal orchestration overhead. 
• Suitability for emergent problem-solving. 

Limitations 

• Limited execution control.
• Weak HITL support.

Best applications

• Coding agents.
• Brainstorming systems.
• AI research experiments.

CrewAI

• Core design: Role-based task delegation.
• Philosophy: Teams of specialized agents.

CrewAI is centered around building simple, structured multi-agent systems. It is similar to AutoGen, modeling AI agents as members of a “crew” where each agent has a clearly defined role. The goal is to make multi-agent systems approachable, even if you are new to agentic AI.

CrewAI prioritizes simplicity and speed over deep memory and production controls, making it easy to learn and a strong option for prototypes and small teams. However, its limited toolset for observability, HITL, and error handling at scale makes it less suited for larger systems.

Strengths

• Very approachable API.
• Clear role separation.
• Fast setup.

Limitations

• Lightweight memory.
• Limited production controls.

Best applications

• Content pipelines.
• Market research automation.
• Simple workflow agents.

Semantic Kernel

• Core design: Planner-based orchestration.
• Philosophy: Enterprise-grade AI integration.

Semantic Kernel is another open-source Microsoft framework, designed for building AI-powered applications that integrate with existing enterprise systems.

It was created with production concerns in mind from the start, emphasizing governance, safety, observability, and human oversight. Rather than maximizing agent autonomy, it focuses on making AI predictable, controllable, and auditable.

By combining structured workflows with LLM reasoning, it trades flexibility and emergent behavior for trust, safety, and operational reliability.

Strengths

• Strong HITL support.
• Enterprise-friendly architecture.
• Good observability.

Limitations

• Heavier upfront structure.
• Less flexibility for open-ended autonomy.
• Steeper learning curve.

Best applications

• Internal enterprise tools.
• AI copilots.
• Business process automation.

smolagents

• Core design: Minimalist chain-based.
• Philosophy: Simplicity over scale.

smolagents is a bare-bones framework designed to make agentic AI as straightforward and transparent as possible. It prioritizes simple, readable code that makes it easy to understand how an agent works without needing to learn a large framework.

smolagents aims to make agent behavior accessible and easy to experiment with by keeping abstractions minimal and logic transparent. It offers first-class support for code-based and tool-calling agents, broad model and tool compatibility, and lightweight CLI utilities, while intentionally trading large-scale orchestration and production features for simplicity and clarity.

Strengths

• Extremely lightweight design.
• High degree of transparency.
• Fast experimentation.

Limitations

• Limited suitability for scaling
• Minimal production features.

Best applications

• Educational projects.
• Proofs of concept.
• Lightweight local agents.

OpenAI Agents SDK

• Core design: Managed workflow-driven orchestration (often graph-based).
• Philosophy: Hosted, production-ready agents.

Thanks to ChatGPT’s explosion in popularity, we’ve all heard of OpenAI. The Agents SDK is the company’s effort to provide a managed platform for building and running agents without having to maintain your own orchestration infrastructure.

Rather than assembling agents from scratch, you define agent behavior and workflows, while OpenAI provides orchestration, memory management, monitoring, and safety controls. This makes the Agents SDK particularly attractive for teams that want production-ready agents quickly.

Strengths

• Minimal infrastructure burden.
• Built-in safety and observability.
• Strong multi-agent support.

Limitations

• Reduced customization and control.
• Limited suitability for experimental research.

Best applications

• SaaS agent features.
• Customer-facing autonomous systems.
• Teams prioritizing speed over customization.

Phidata

• Core design: Agent-centric, tool-heavy.
• Philosophy: Practical agents for real-world data tasks.

Phidata is designed for building practical, tool-driven AI agents that operate on real-world data.

Rather than focusing on abstract orchestration patterns, Phidata centers the agent around direct interaction with systems such as APIs, databases, and internal services.

Its design reflects the fact that many agents spend most of their time fetching, transforming, and acting on data.

Strengths

• Strong tool integration.
• Suitability for data-centric workflows.

Limitations

• Less emphasis on orchestration.
• Limited multi-agent capabilities.

Best applications

• Data analysis agents.
• Finance and ops automation.
• Tool-driven decision systems.

Choosing the right framework

Now that you’re familiar with many of the most popular frameworks in 2026, it’s time to choose the right one for your project. Let’s take a look at some of the key use cases, along with the frameworks that fit them best.

In 2026, agentic frameworks have evolved from experimental tools into foundational infrastructure for many applications. The key decision is no longer whether to use agents, but how much control, autonomy, and governance your systems require.

In this tutorial, we build a live object detection app using TensorFlow and PyCharm, then deploy it onto the Reachy Mini open-source robot for real-time object tracking.

Reachy Mini is a compact open-source robot built in collaboration by Pollen Robotics, Hugging Face, and Seeed Studio. It has been going viral lately, getting mentioned in NVIDIA videos and even in the keynotes at some of their conferences. What makes it particularly interesting is that not only is all the code open-source, the body is too, which means you can print your own parts and develop your own apps to run on it.

There is an app store of community-built projects you can explore and try, and easily contribute to. Anything conversational or camera-based is especially fun to build because of the hardware it ships with: a speaker, a microphone, and a camera, plus expressive antennas for emotions.

This really highlights the unique new type of robot that the Reachy Mini embodies: It almost feels like it is a physical representation of an LLM or an AI agent, rather than a robot that has AI added to it. It does not have a body that moves around or hands to grab things, so its main selling point is really its brain. That design choice shapes what is most interesting to build with it.

Let’s learn how to build a TensorFlow object detection app and deploy it on the Reachy Mini, which will then allow us to do live object tracking. You can head over to the PyCharm channel for the full code breakdown and try it at home. All the code is in the Reachy-mini-object-detection GitHub repository.

For an introduction to the robot, you can first watch Iulia’s video here:

What you’ll learn

• How to build a real-time TensorFlow object detection pipeline. 
• How to use SSD MobileNet V2 from TensorFlow Hub. 
• How to create a TensorFlow object detection example with OpenCV. 
• How to run live webcam inference in PyCharm notebooks. 
• How to deploy object detection on the Reachy Mini robot. 
• How to track detected objects using head movement logic. 
• How to stream annotated detections to a live dashboard.

What we are building

The project is split into two stages.

Stage 1 is a standalone notebook that runs entirely on your laptop using your webcam. No robot needed. This is where we make sure the detection pipeline works correctly before touching any hardware.

Stage 2 is a Reachy Mini app that integrates the same model with the robot: Her head moves to follow detected objects, her antennas wiggle when she spots something new, and a live web dashboard at http://0.0.0.0:8042 shows the annotated camera feed and detections.

You can follow along with the step-by-step video tutorial:

How TensorFlow object detection works: Step by step

1. Capture an image frame from the webcam. 

2. Convert the frame into a TensorFlow tensor.

3. Run inference through the pretrained model.

4. Receive bounding boxes, labels, and confidence scores.

5. Filter low-confidence detections.

6. Draw annotated results onto the frame.

7. Display the processed image in real time.

Prerequisites

• Python 3.12+.
• PyCharm with its Jupyter Notebook integration.
• A Reachy Mini for Stage 2 (Stage 1 runs entirely on your laptop).
• Some familiarity with TensorFlow basics – if you are brand new to it, the previous post in this series is a good starting point.

Stage 1: Building a Tensorflow object detection pipeline in PyCharm

Before connecting the robot, we want to make sure the TensorFlow part works independently. We are going to make a notebook that only executes through our object detection model and makes it run smoothly. PyCharm’s native notebook integration is a great fit here: You can inspect each step of the pipeline and visualize results inline.

The object detection model

We are using SSD MobileNet V2 from TensorFlow Hub, trained on Open Images V4. This popular model from Google provides SSD-based object detection and has been trained on a lot of open images. With a little bit of fine-tuning you can deploy it with your own use case, though for this tutorial, the general model works well without any fine-tuning at all.

It runs at around 10 FPS on CPU, which is fast enough for responsive real-time behavior on the robot.

Install dependencies

!pip install tensorflow tensorflow-hub opencv-python numpy Pillow

Load the model

import tensorflow as tf
import tensorflow_hub as hub
import numpy as np
import cv2
import time
from IPython.display import display, clear_output
from PIL import Image

MODEL_HANDLE = "https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1"

print(f"TensorFlow version: {tf.__version__}")
print("Loading model (first time downloads ~30MB)...")

detector = hub.load(MODEL_HANDLE)
print("Model loaded!")

The model is about 30 megabytes and gets cached locally after the first download. Because it is very generalized, it can work across a lot of different scenarios without needing additional training data, which makes it a lot easier to get started.

Detection and drawing helpers

We need two helper functions: one to run inference and return a list of detections, and one to draw the bounding boxes on the frame. These are the same functions we use later in the Reachy app.

def detect_objects(frame_bgr, min_score=0.5, max_detections=10):
    rgb = frame_bgr[:, :, ::-1]
    img_tensor = tf.image.convert_image_dtype(rgb, tf.float32)[tf.newaxis, ...]

    results = detector.signatures['default'](img_tensor)

    boxes = np.array(results["detection_boxes"])
    scores = np.array(results["detection_scores"])
    class_labels = np.array(results["detection_class_entities"])

    if boxes.ndim > 2:
        boxes = boxes[0]
    if scores.ndim > 1:
        scores = scores[0]
    if class_labels.ndim > 1:
        class_labels = class_labels[0]

    scores = np.atleast_1d(scores)
    indices = [i for i, score in enumerate(scores) if score >= min_score][:max_detections]

    detections = []
    for idx in indices:
        ymin, xmin, ymax, xmax = boxes[idx]
        label = class_labels[idx].decode('utf-8') if isinstance(class_labels[idx], bytes) else str(class_labels[idx])
        detections.append({
            "box": [ymin, xmin, ymax, xmax],
            "score": float(scores[idx]),
            "label": label
        })

    return detections


def draw_detections(frame_bgr, detections):
    h, w = frame_bgr.shape[:2]
    annotated = frame_bgr.copy()

    for det in detections:
        ymin, xmin, ymax, xmax = det["box"]
        x1, y1 = int(xmin * w), int(ymin * h)
        x2, y2 = int(xmax * w), int(ymax * h)

        color = (0, 255, 0)
        cv2.rectangle(annotated, (x1, y1), (x2, y2), color, 2)

        label = f"{det['label']} {det['score']:.0%}"
        font_scale, thickness = 0.6, 2
        (tw, th), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, font_scale, thickness)
        cv2.rectangle(annotated, (x1, y1 - th - 8), (x1 + tw + 4, y1), color, -1)
        cv2.putText(annotated, label, (x1 + 2, y1 - 4),
                    cv2.FONT_HERSHEY_SIMPLEX, font_scale, (0, 0, 0), thickness)

    return annotated

The detect_objects function runs inference using the model’s detect_objects entry point and handles flattening the batch dimension from the output tensors. Labels come back as bytes from the model, so we decode them to strings before returning.

Test on a single frame

cap = cv2.VideoCapture(0)
ret, frame = cap.read()
cap.release()

if not ret:
    print("ERROR: Could not access webcam. Make sure no other app is using it.")
else:
    print(f"Frame captured: {frame.shape}")

    t0 = time.time()
    detections = detect_objects(frame)
    elapsed = time.time() - t0

    print(f"Inference time: {elapsed:.2f}s ({1/elapsed:.1f} FPS)")
    print(f"Found {len(detections)} objects:")
    for d in detections:
        print(f"  - {d['label']}: {d['score']:.0%}")

    annotated = draw_detections(frame, detections)
    display(Image.fromarray(annotated[:, :, ::-1]))

This is the stage where you check that the model is detecting correctly and the bounding boxes are drawn in the right places. The inline image display in PyCharm’s notebook view makes it easy to see the result right there in the cell.

Running real-time TensorFlow object detection with OpenCV

Once the single-frame test looks good, you can run it continuously:

cap = cv2.VideoCapture(0)

if not cap.isOpened():
    print("ERROR: Could not open webcam.")
else:
    print("Running live detection... (interrupt kernel to stop)")
    try:
        while True:
            ret, frame = cap.read()
            if not ret:
                break

            t0 = time.time()
            detections = detect_objects(frame)
            fps = 1.0 / max(time.time() - t0, 0.001)

            annotated = draw_detections(frame, detections)
            cv2.putText(annotated, f"{fps:.1f} FPS", (10, 30),
                        cv2.FONT_HERSHEY_SIMPLEX, 1.0, (0, 0, 255), 2)

            clear_output(wait=True)
            display(Image.fromarray(annotated[:, :, ::-1]))

            labels = ", ".join(f"{d['label']} ({d['score']:.0%})" for d in detections)
            print(f"{fps:.1f} FPS | {len(detections)} objects: {labels or 'none'}")

    except KeyboardInterrupt:
        print("Stopped.")
    finally:
        cap.release()
        print("Camera released.")

At this point we have built a notebook that works with just having object detection and we can use this with a simple camera of whatever type you have around. Now, we can wrap it up and make it into an app that we can deploy on the Reachy.

Stage 2: Deploying the TensorFlow object detection app on the Reachy Mini

The Reachy Mini app lives in the reachy_mini_object_detector/ folder and extends the detection logic with head tracking, antenna reactions, and a web dashboard. We’ve followed the guidelines for building Reachy Apps laid out in this blog post. Particularly, we can leverage a helper LLM system like Claude by giving it the predefined Agent Helper documentation.

Project structure

reachy_mini_object_detector/
├── pyproject.toml
└── reachy_mini_object_detector/
    ├── detector.py       # TF Hub model wrapper
    ├── main.py           # App: head tracking + web dashboard
    └── static/           # Web UI assets (served at :8042)

The detector.py file wraps the model and the detect_objects logic. main.py imports from it and adds everything specific to the robot.

Installing the app

From the Reachy Mini dashboard, under Apps, or by manually adding:

pip install git+https://huggingface.co/spaces/backtoengineering/reachy_mini_object_detector

How head tracking works

The app runs two loops in parallel: an inference thread that grabs frames from the robot’s camera and runs detection, and a main control loop at around 50Hz that handles head movement and antenna control.

The head tracking feature maps the detected object’s position in the frame to a yaw and pitch offset for the head. The camera has a horizontal field of view of 60 degrees and a vertical field of view of 45 degrees. When an object is at the center of the frame its center_x is 0.5, so subtracting 0.5 and multiplying by the field of view gives the angle offset to track it:

target_yaw = -(largest.center_x - 0.5) * CAMERA_FOV_H_DEG
target_pitch = (largest.center_y - 0.5) * CAMERA_FOV_V_DEG

Rather than snapping the head instantly to that target, the app uses a smoothing factor (TRACKING_ALPHA = 0.15) so the movement looks natural:

self._current_yaw += TRACKING_ALPHA * (target_yaw - self._current_yaw)
self._current_pitch += TRACKING_ALPHA * (target_pitch - self._current_pitch)

When nothing is detected, the head slowly drifts back toward center rather than freezing in place.

Antenna wiggle

The antennas wiggle when a new object class is first detected, not on every frame. The app keeps track of which classes have already been seen in _seen_classes, and when something new appears it sets a wiggle timer for 1.5 seconds. During that window, the control loop drives a sinusoidal antenna movement as follows:

phase = (t - t0) * 8.0  # fast wiggle
antenna_val = np.deg2rad(20.0 * np.sin(phase))
antennas = np.array([antenna_val, -antenna_val]

This makes the interaction feel intentional: Reachy reacts when she sees something new, rather than wiggling constantly while tracking.

The web dashboard

The app serves a live dashboard (available at http://0.0.0.0:8042) with the annotated camera feed (as an MJPEG stream), the current detection list, an FPS counter, and a toggle to enable or disable head tracking. This is useful during development because you can see exactly what the model is detecting from the robot’s perspective in real time.

Where to go next

This is a great starting point and there are a lot of directions you can take it:

• Run the app with a specific use case in mind. The model is general, but if you want Reachy to recognize specific objects you can fine-tune on your own dataset using TensorFlow’s Object Detection API.
• Add more apps. There are many apps that users have already created in the Reachy Mini store, and building one that uses both the camera and the conversational capabilities together opens up a lot of possibilities.
• Connect to physical arms. Something I would really like to explore next is connecting Reachy to the SO-101 arms, so she can actually reach out and do things in the physical world as well as see them.

You can find all the code in the Reachy-mini-object-detection repository. Everything is open-source, so feel free to build on it, adapt it, or deploy your own version.

FAQs

What is TensorFlow object detection?

TensorFlow object detection is a computer vision technique that uses machine learning models to identify and locate objects within images or video streams.

What is the best TensorFlow object detection model for real-time applications?

SSD MobileNet V2 is commonly used for real-time TensorFlow object detection because it balances inference speed and accuracy efficiently.

Can TensorFlow object detection run on CPU?

Yes. Models like SSD MobileNet V2 can run entirely on CPU, making them suitable for laptops, edge devices, and robotics projects.

What is the difference between the TensorFlow Object Detection API and TensorFlow Hub?

TensorFlow Hub provides pretrained reusable models, while the TensorFlow Object Detection API offers a larger framework for training, evaluation, and deployment workflows.

Can I train TensorFlow object detection on custom data?

Yes. You can fine-tune pretrained models using your own labelled datasets to detect custom objects.

About the author

Iulia Feroli

Iulia Feroli is the founder of the Back To Engineering community on YouTube, where she builds robots, explores physical AI, and makes complex engineering topics accessible and fun. She has a background in data science, AI, cloud architecture, and open source.

You can use the audio player below to listen to this blog post.

Better compatibility with assistive technologies

One of the key areas we’ve been working on is improving how JetBrains IDEs interact with OS-level accessibility tools.

Improved Magnifier support on Windows

Screen magnifiers are among the most commonly used assistive technologies in JetBrains IDEs. Until recently, the built-in Windows Magnifier didn’t reliably follow the text cursor in the editor, making navigation and editing more difficult for low-vision users. We’ve implemented support for cursor tracking so Magnifier follows text as you type, just as it does in other applications.

This builds on earlier work on macOS, where we addressed text cursor tracking with macOS Zoom. Now, the same support is being extended to Windows.

Orca and GNOME Magnifier support on Linux

With version 2026.2, coming this summer, JetBrains IDEs will allow you to use the Orca screen reader and GNOME Magnifier in supported Linux environments. 

This is an active area of work, with multiple related tasks already underway. Accessibility shouldn’t depend on your operating system, and we’re continuing to improve support across platforms.

More predictable keyboard navigation

We’ve also been making it easier to move through the IDE without relying on a mouse.

Main menu access with Alt on Windows

In native Windows applications, pressing Alt moves the focus to the main menu, allowing you to navigate it with the keyboard. This behavior was previously missing from JetBrains IDEs, and screen readers, such as NVDA, would sometimes announce the system menu instead.

Now, the main menu behaves in a way that feels familiar and predictable for keyboard-only and screen-reader users, and the bright focus indicator helps low-vision users identify the selected item.  

Navigating between major parts of the IDE

Another focus area is the experience of moving between different parts of the IDE interface, such as toolbars, panels, and the editor. We’re working on a more structured model for navigating through the big component groups:

• Tab and Shift+Tab move the focus within the current area.
• A dedicated shortcut lets you jump between larger sections of the IDE.

This reduces the effort required to reach essential controls and makes the overall layout easier to navigate. For the current iteration, we made it possible to bring the main toolbar and status bar into focus, and we fixed the Project and Git toolbar widgets, which were not selectable by screen readers, even though other elements already were. 

As the next step, we’ll polish specific controls and include tool window bars on both sides of the IDE frame in the navigation flow.

Exploring richer non-visual feedback with audio cues

Accessibility is not only about reaching controls, but also about understanding what’s happening while you work. We’re exploring ways to provide richer audio feedback in the IDE. Two directions we’re currently investigating:

• Contextual signals when the caret lands on lines with errors, warnings, breakpoints, or version control changes. We want the IDE to provide immediate, non-visual feedback in context.
• More general audio notifications for IDE actions and state changes.

The goal is to reduce the need to rely on visual indicators or switch contexts just to understand what changed. Instead, we want the IDE to provide that information more directly.

Accessibility as an ongoing effort

We’re improving accessibility in JetBrains IDEs across multiple areas at once, including by providing compatibility with assistive technologies like screen readers and magnifiers, as well as by offering more consistent keyboard navigation and clearer feedback for events that are otherwise mostly visual.

These improvements build on earlier updates, such as support for VoiceOver and NVDA, a high-contrast UI theme, and color schemes for red-green vision deficiency. There’s still more to do, and we’ll continue working in this direction.

We’d love to hear from you

We’re eager to hear from developers who rely on accessibility features, as well as from anyone interested in improving the experience of using them.

If you have ideas or feedback about accessibility in JetBrains IDEs, you can reach us directly at accessibility@jetbrains.com. You can also report issues through YouTrack or the support request form.

If you’d like to stay informed about accessibility improvements, you can subscribe to updates here.

Artificial intelligence keeps evolving at a rapid pace. The latest major application of AI, specifically of LLMs, is AI agents. These are systems that use their perception of their environment, processes, and input to take action to achieve specific goals, and they are built on LLMs. 

Increasingly, complex AI agents are being used in real-world applications. While simpler agentic applications that use only one agent to achieve a goal still exist, organizations are now shifting towards multi-agent systems that use multiple subagents coordinated by a main agent. These are more adaptable and can mimic human teams when it comes to performing specialized tasks such as data analysis, compliance, customer support, and more. The reasoning and autonomy of AI agents have improved; consequently, they can gather data, conduct cross-references, and generate analysis.

As we move towards these complex, real-world applications of agents, an ever-stronger spotlight is being shone both on how we observe AI agents and how we evaluate the LLMs they’re built upon. The complexity, interactions, and autonomous processes under the surface of AI agents make rigorous monitoring and assessment an essential part of building and maintaining these applications. LLM evaluation determines if the AI agent can work, while AI agent observability determines if it is working. LLM evaluation tests an agent’s basic capabilities before and during deployment, while agent observability provides deep, real-time visibility into an agent’s internal reasoning and operational health once it is live. It is pretty obvious that having just one of these is a loss and a formula for failure. 

In this blog post, we’ll explore how to evaluate agents using advanced metrics and observability tools. It’s designed as a practical, end-to-end reference for teams that want to move beyond demos and actually run AI agents in live, real-world environments, avoiding the common pitfalls that cause failure in production.

Core LLM evaluation metrics for modern AI systems

As LLMs are now applied to a wide range of use cases, it is important that their evaluation covers both the tasks they may perform and their potential risks. Evaluation metrics give a better understanding of the strengths and weaknesses of LLMs, influence the guidance of human-LLM interactions, and highlight the importance of ensuring LLM safety and reliability. Hence, LLM evaluation metrics for assessing the performance of an LLM are indispensable in modern AI systems. Without well-defined evaluation metrics, assessing model quality becomes subjective. 

There are several key evaluation metrics, each with a different purpose, and the table below provides a summary of some of them.

Hallucination rate

Hallucinations in LLMs produce outputs that seem convincing yet are factually unsupported and can be categorized as either intrinsic, where the output contradicts the source content, or extrinsic, where it simply cannot be verified. They can stem from a range of factors across data, training, and inference, from quality issues in the large datasets used for initial training and the data used to fine-tune model behavior to post-training techniques that make models overly eager to provide responses to imperfect decoding strategies at inference. Because hallucination is an unsolved challenge cutting across every stage of model development, measuring and assessing it remains a vital part of LLM evaluation.

There is a wide variety of techniques for detecting hallucinations. These include: 

• Fact-checking: Extracting independent factual statements from the model’s outputs (fact extraction) and then verifying these against trusted knowledge sources (fact verification).
• Uncertainty estimation: Using the certainty provided in the model’s internal state to estimate how likely a piece of factual content is to be a hallucination.
• Faithfulness hallucination detection: Ensures the faithfulness of LLMs to provide context or user instructions. 

There are several metrics for hallucination detection. Some of the most commonly used metrics include:

• Fact-based metrics: Assessing faithfulness by measuring the overlap of facts between the generated content and the source content. 
• Classifier-based metrics: Utilizing trained classifiers to distinguish between the level of entailment between the generated content and the source content. 
• QA-based metrics: Using question-answering systems to validate the consistency of information between the source content and the generated content. 
• Uncertainty-based metrics: Assessing faithfulness by measuring the model’s confidence in its generated outputs. 
• LLM-based metrics: Using LLMs as evaluators to assess the faithfulness of generated content through specific prompting strategies. 

PyCharm’s Hugging Face integration lets you discover evaluation models and datasets without leaving the IDE. Use the Insert HF Model feature to search for hallucination or toxicity classifiers, and hover over any model or dataset name in your code to instantly preview its model card, including training data, intended use, and limitations. This means you can import a dataset, evaluate your LLM, and verify the tools you’re using, all from one place.

Trust is imperative in the acceptance and adoption of technology. Trust in AI is especially important in areas such as healthcare, finance, personal assistance, autonomous vehicles, and others. Hallucinations have a huge impact on users’ trust in LLMs.

In 2023, a story went viral about a Manhattan lawyer who submitted a legal brief largely generated by ChatGPT. The judge quickly noticed how different it was from a human-written submission, revealing clear signs of hallucination. Incidents like this highlight the real-world risks of LLM errors and their impact on user trust. As people encounter more examples of hallucination, skepticism around LLM reliability continues to grow.

Toxicity scores

LLMs that have been pretrained on large datasets from the web have the tendency to generate harmful, offensive, and disrespectful content as well as toxic language, such as hate speech, harassment, threats, and biased language, which have a negative impact on their safe deployment. Toxicity detection is the process of identifying and flagging toxic content by integrating open-source tools or APIs into the LLM workflow to analyze both the user input and the LLM output. Some of the available toxicity tools include the OpenAI Moderation API, which is free, works with any text, and has a quick implementation. Perspective API by Google is also widely used with a transparent methodology, but will no longer be in service after 2026. Detoxify, which is open source, has no API costs, and is Python-friendly, and Azure AI Content Safety by Microsoft, which is customizable and best for enterprise deployments and existing Azure users. Hugging Face Toxicity Models have many model options and easy integration with Transformers.

Toxicity detection has become a guardrail; hence, it is important in public-facing applications. They prevent toxic content from reaching users, which protects both individuals and organizations. In public-facing applications, toxicity detection operates by input filtering, output monitoring, and real-time scoring. This prevents attacks where users intentionally train AI to produce toxic content through coordinated toxic inputs; toxic content will never reach the user, even if produced by the underlying AI, so systems can adjust their behavior dynamically based on conversation content and escalating risks. Unguarded AI can be exploited, which leads to reputational damage. 

For toxicity evaluation, PyCharm’s Hugging Face Insert HF Model feature helps you discover classifiers like s-nlp/roberta_toxicity_classifier directly in the IDE. Hovering over the model name reveals its model card, where you can see it was trained on the Jigsaw toxic comment datasets, helping you understand what the model can and can’t detect before you write a single line of evaluation code. 

Frameworks for LLM evaluation

Frameworks for LLM evaluation have changed the game; teams don’t have to rely on manual reviews, gut instinct, and subjective judgment to assess model quality. These frameworks automate the measurement of model quality using standardized, quantifiable metrics. They assign numerical scores to outputs that measure faithfulness, relevancy, toxicity, and other important dimensions. This automation results in reproducibility, speed, and objectivity. 

Consequently, the same input always produces the same score; evaluation runs 10–100 times faster, so in minutes instead of days; and there are no more debates on the quality of the output. Some of these frameworks include DeepEval and Retrieval Augmented Generation Assessment (Ragas). DeepEval is an open-source evaluation framework built with seven principles in mind, such as the ability to easily “unit test” LLM outputs in a similar way to Pytest and plug in and use over 50 LLM-evaluated metrics, most of which are backed by research and all of which are multimodal. 

It is extremely easy to build and iterate on LLM applications with two modes of evaluation, namely, end-to-end LLM evals and component-level LLM evals. It is used for comprehensive testing across RAG, agents, and chatbots. Ragas is a framework for reference-free evaluation of RAG pipelines. There are several dimensions to consider, such as the ability of the retrieval system to identify relevant and focused context passages, as well as the capability of the LLM to exploit such passages in a faithful way; hence, it is challenging to evaluate RAG systems. Ragas provides a suite of metrics for evaluating these dimensions without relying on ground-truth human annotations. 

The limits of static prompt evaluation

Traditional LLM evaluation methods are useful for single prompt-response pairs, measuring output quality, RAG systems with straightforward retrieval, and static evaluation with fixed inputs. But they are limited for multi-step agents because LLM evaluation focuses on the final output quality, not the decision-making process that produced it. Multi-step agents exhibit a different kind of complexity, as they chain multiple decisions.

Why traditional LLM evaluation isn’t enough for agents 

Agents operate independently within complex workflows, and this independence can introduce challenges such as deviation from expected behavior, errors in production, and more failure points than in traditional software applications. Hence, an agent can perform well in testing but fail in production. Traditional LLM evaluations don’t have the capacity to test such use cases. Testing is usually done in a controlled environment with limited scenarios, but production involves real users, edge cases, unpredictable inputs, and scale. This means that agents can make decisions that are not seen in testing, and in production, tasks could be completed, though incorrectly, without generating an error signal. This is where advanced evaluation and monitoring practices come to the rescue! They provide the visibility and systematic measurement needed to deploy agents confidently, rather than relying on trial and error.

The complexity of agent behavior

Traditional LLM evaluation measures single prompt-response pairs: provide an input prompt, receive an output response, and measure quality through metrics such as accuracy, relevance, and faithfulness. Due to the complexity and non-deterministic, multi-step reasoning of AI agents, they cannot be reliably evaluated using traditional evaluation metrics.

Agent behavior is complex, and this complexity introduces challenges. Agents operate in dynamic environments where APIs might be down, databases change between queries, and the “right” answer depends on current conditions. They can use external tools and APIs to complete tasks, and may either use the wrong tool or use the right tool with the wrong parameters or input type. Their internal reasoning traces remain hidden unless they are logged explicitly, so it might be challenging to determine whether an agent was successful through logic or chance. An agent’s output could be perfectly correct despite poor internal decisions, or the entire task could fail despite correct step execution.

This is where observability tooling becomes essential. PyCharm’s AI Agents Debugger breaks open the black box of agentic systems, letting you trace LangGraph workflows and inspect each agent node’s inputs, outputs, and reasoning directly in the IDE, with zero extra code. Just install the plugin, run your agent, and the debugger automatically captures execution traces. Click the Graph button to visualize the full workflow, making it easy to spot where an agent chose the wrong tool, passed bad parameters, or succeeded by luck rather than logic.

To see this in action, I built a simple travel-planning agent using LangGraph in two steps: a research node that suggests summer destinations based on my preferences, and a plan node that picks the best option and builds a three-day itinerary. With the AI Agents Debugger, you can trace exactly what information flowed between these two steps – what the research node suggested and how the planner used those suggestions to build the final itinerary.

Advanced agent evaluation metrics

The complexity of AI agents demands evaluation that goes beyond considering the final output quality, that is, measuring whether it is accurate, relevant, and grounded. Specialized agent evaluation assesses the complete decision-making process, including the planning logic, tool selection, parameter construction, reasoning coherence, and resource efficiency that led to the final output. Hence, the advanced agent evaluation metrics are designed to make such a process visible and measurable. Some of them are task completion rate, tool usage, reasoning quality, efficiency, and error handling.

Task completion rate

Task completion rate measures the percentage of tasks where an agent successfully achieves the end goal. This is calculated as the number of completed tasks divided by the total number of tasks attempted. The context of “completed” differs by use case. There are real-world use cases for task completion rate. Let’s start with a basic use case. Consider a customer service agent handling a specific food delivery order: “Where is my order #0001? It has not been delivered to me.” Completion rate means successfully looking up the order ID, retrieving the tracking information, and providing an accurate delivery estimate, so all three steps must succeed. If the agent retrieves the wrong order or fails to assess the tracking system, that is a failed task, even if it produces the same output. 

Next, let us look at a medium-complexity use case, sequential API calls. Consider an agent tasked with creating a Jira support ticket and notifying the relevant team in Slack. The agent calls the Jira API to create a ticket, parses the response to get the ticket ID, calls the Slack API with the ticket link, and finally verifies the success of both. If the agent successfully creates the Jira ticket, but the Slack notification fails, that is considered a failed task even if the ticket exists in Jira, since the team wasn’t notified. 

Finally, let’s examine a high-complexity use case: An agent is given the task of completing an online purchase, which means it must handle everything from checkout to order confirmation. Six steps are involved: Verify the item is still in stock, process the payment with a credit or debit card, reserve or decrement inventory, create an order record, generate an order confirmation number, and send a confirmation email to the customer. If the agent successfully charges the customer’s card but the confirmation email fails to send, that’s a failed task, even if the payment was processed and the order was created. In such a situation, the customer has no proof of purchase, so they will likely contact support or attempt to purchase again.

Tool usage correctness

Tool usage correctness assesses whether an agent correctly identifies and invokes the relevant tools and APIs. It is a deterministic measure that is assessed using techniques such as LLM as a judge, like most LLM evaluation metrics. It has three dimensions: 

• Did the agent choose the right tool for the task (tool selection)? 
• Were the parameters constructed correctly (input parameters)? 
• Did the agent properly use the tool results (output handling)? 

Hence, it is important for reliability and functional correctness. 

Step-by-step reasoning accuracy

In real-world use cases, an LLM agent’s reasoning is shaped by much more than just the model itself. Modern frameworks such as LangChain expose the agent’s internal “thoughts” through structured logging of intermediate reasoning steps. This is done using the ReAct (Reasoning and Acting) pattern, which involves the agent thinking about what to do, using a tool, observing the tool result, and then repeating until the task is complete. Each “thought” is logged as text, which creates a complete trace of the reasoning process from initial query to final answer. These traces can be extracted programmatically and evaluated to assess whether the agent’s logic is sound even when the final output appears correct. Evaluating planning steps involves assessing aspects such as the overall approach’s logic, the ordering of steps, and whether any steps are unnecessary or redundant. Evaluating execution assesses whether the implementation worked, such as whether tools were called with correct parameters, whether each step was completed successfully, whether errors were handled appropriately, and whether the output was interpreted correctly. This can be done seamlessly in PyCharm using the AI Agents Debugger.

Groundedness (faithfulness)

Groundedness, also known as faithfulness, is the most critical metric for retrieval-augmented generation (RAG), which is a common component of agentic applications. It assesses whether the agent’s response is actually supported by the retrieved source documents or whether, instead, the model hallucinated information. Different evaluation techniques include:

• Atomic claim verification: Breaks up the response into atomic claims and checks each claim against the retrieved context. It is slow but best for production RAG and thorough evaluation. 
• Semantic similarity: Compares the embeddings of the response and source documents. It is fast, so it is best for quick checks and first-pass filtering. 
• LLM-as-Judge: works by prompting the LLM to score groundedness by extracting factual statements from the response and then checking each statement against the retrieved context. It offers medium speed and is best for flexible, custom criteria. 

AI observability and why it matters

AI observability is about visibility into what the agent is doing. This covers recording everything that happens when a task is executed, including the agent’s reasoning at each step, which tools were called with what parameters, what data was retrieved, and how decisions were made from start to finish. With such a transparent system where every decision can be logged and traced, teams are able to understand why an agent fails, behaves unexpectedly, or becomes expensive to run because issues can be debugged and behavior can be audited. Consequently, system design improves, and guesswork is eliminated.

Definition of AI observability

AI observability is the real-time monitoring of agent actions, thoughts, and environmental interactions: what went in, what came out, how the agent thought through the problem, and which tools, APIs, and data were used. AI observability builds on the three pillars of DevOps observability – that is, metrics, logs, and traces – but extends each one for AI’s unique needs. DevOps metrics track CPU and latency, while AI metrics track token usage and cost per interaction. DevOps logs capture system errors, while AI logs capture reasoning traces and decision points. DevOps traces follow requests through services, while AI traces follow reasoning through agent steps, tool calls, and observations.

Benefits for agent monitoring

Agent monitoring has immense benefits – here are some of the most important:

• It debugs reasoning errors: When an agent fails or gives an unexpected output, monitoring provides a complete trace of its decision-making process, which shows exactly where the logic broke down. Hence, there is no need to spend hours guessing the causes.
• It measures performance and latency over time: Since metrics such as average latency, token usage, cost per interaction, and completion rates across all queries are tracked, degradation patterns can be identified before they affect users. As a result, performance issues can be identified and resolved before users file any complaints. 
• It identifies regressions after model or prompt updates: Baseline metrics such as completion rate, faithfulness scores, latency, and cost are established and then monitored for deviations after deployments. If a new prompt drops the compilation rate or a model update increases the hallucination rate, automated alerts catch it immediately. Hence, issues are caught before users are affected.

Popular tools for agent monitoring

Several frameworks and platforms have emerged to provide built-in observability for AI agents, with each having different strengths and integration approaches and matching different features and requirements. The choice of the right tool depends on the framework, deployment preferences, and primary needs. The table below shows some popular tools and whether they match different features and requirements.

Best practices for evaluating agents in production

Evaluation does not end after deployment; rather, it is intensified. This continuous evaluation tracks how much the system costs to run, how quickly it responds under various loads, and how it handles errors or unusual inputs. Without such evaluation, problems can only be identified after the users are affected. An agent can pass all the quality checks with excellent faithfulness scores, high completion rates, and strong reasoning but fail in production if costs spiral, latency increases, or edge cases cause instability. Hence, there is a critical need for ongoing evaluation and monitoring, which will lead to systems that are reliable, scalable, and financially sustainable.

Monitor cost and latency

Monitoring cost and latency is critical for production sustainability. Token usage and response time must be tracked continuously because small inefficiencies compound dramatically over time, and the cost per token of the powerful reasoning models used for agents can be high. Production workloads require cost and latency monitoring to identify problems before user experience and budget are impacted. Cost monitoring tracks token usage at different levels, such as per request, per query type, and over time. Without visibility into patterns generated by these, teams end up discovering cost problems through surprise bills. With monitoring, they can proactively cache common queries and optimize prompts to reduce token use. Latency monitoring reveals track response time and component breakdowns to identify bottlenecks.

Cost control in production workloads is important because production costs can spiral quickly, unmonitored systems can exceed budgets, and latency impacts user experience and retention.

Combine offline and online evaluation

Effective agent evaluation requires combining offline and online evaluation, where each addresses gaps the other leaves. Offline evaluation uses fixed test databases for reproducible benchmarking, which enables fast iteration on prompts and models in controlled environments without production risk. Online evaluation monitors real user interactions in production, which reveals edge cases in testing that were never expected, so it is useful for real-time feedback, user data, and observability tools. A combination of both results in an optimal strategy where offline evaluation validates changes before deployment, then online evaluation monitors production reality. 

Use human-in-the-loop when necessary

LLM agents are appreciated for how they have played a positive role in the different ecosystems, but not every agent should run autonomously since they can misinterpret prompts, cross boundaries, or make dreadful errors that can’t be caught by automation alone. Hence, the need for human-in-the-loop failsafes. Human-in-the-loop is also essential during initial setup: Unless teams already have domain-specific evaluation datasets for monitoring the agent, these will need to be created manually by assessing the agent’s performance. A hybrid approach is required when critical decisions require human validation, such as approving transactions, modifying sensitive data, or triggering irreversible workflows. In this approach, it is important that decisions are routed through a human checkpoint before proceeding. The intention is not to slow automation but rather to ensure that the right decisions involve the right oversight. A well-designed human-in-the-loop system delivers compound returns over time. Every human correction becomes feedback, which improves the agent’s accuracy and gradually reduces the need for manual review. Human oversight isn’t treated as a failure but rather as a safety net that makes the system better with use.

Final thoughts

Fundamentally, AI agents are different from single-prompt LLMs. They navigate multi-step workflows, make autonomous decisions, and use external tools, which introduces complexities that demand continuous evaluation, not just static testing. Evaluation must evolve from pre-deployment checkpoints to ongoing monitoring. Production-ready agents aren’t just well-tested; they’re continuously observed and improved based on real behavior. LLM evaluation and AI observability enable faster, safer iteration by catching issues early and feeding production insights back into development.

PyCharm streamlines agent development with integrated debugging, profiling, and testing. Step through reasoning with breakpoints, find cost bottlenecks, and iterate on evaluation tests rapidly. These workflows transform hours of debugging into minutes of systematic investigation. Explore PyCharm for AI development to see how integrated tools can help you build, evaluate, and deploy reliable AI agents.

About the author

Naa Ashiorkor

Naa Ashiorkor is a data scientist and tech community builder. She is deeply involved in the Python community and serves as an organizer for various conferences, including EuroPython. She is currently building PyLadies Tampere.

What is the Pyrefly LSP?

“LSP” stands for the Language Server Protocol – a standardized protocol that allows code editors and IDEs to communicate with language servers. The LSP enables language servers to provide code intelligence features, such as:

• Code completion
• Information on hover (for example, quick documentation)
• Go to definition and other actions
• Error checking and type-related diagnostics

The key benefit of the LSP is that it allows a single language server to be used across multiple tools. This means that language-specific intelligence does not have to be implemented separately in every editor, IDE, or CI pipeline.

Pyrefly is Meta’s next-generation Python type checker, engineered from the ground up in Rust to replace its predecessor, Pyre (written in OCaml). With the move to Rust, Pyrefly achieves significantly faster performance and improved cross-platform portability. More than just a rewrite, it is designed to be more capable and robust, offering an efficient toolset for maintaining large-scale Python codebases with high precision and minimal overhead.

Pyrefly provides the following benefits:

• Higher performance and efficiency – Thanks to its Rust-based architecture, Pyrefly achieves significantly faster speeds and improves cross-platform portability. 
• Enhanced code intelligence – As an external type provider, Pyrefly powers essential code insight features in the IDE, including type inference, type-related diagnostics, quick documentation, and inlay hints.
• Scalability – Pyrefly is designed to handle large-scale Python codebases with high precision and minimal overhead.

Pyrefly is highly beneficial for projects and developers dealing with large, complex Python codebases that prioritize performance and robust typing. Integrating Pyrefly via the LSP is part of our ongoing work to enhance code insight performance in PyCharm.

Using Pyrefly in PyCharm

Once enabled, Pyrefly powers all code insight functionality in PyCharm, including type inference and type-related diagnostics, quick documentation, and inlay hints. Delegating analysis to this faster engine delivers significantly improved performance.

To start using Pyrefly in your PyCharm project, go to the Type widget at the bottom of the window. By default, the IDE uses the built-in type engine. Click on the widget and select the option to use Pyrefly. If you do not have Pyrefly installed yet, PyCharm will install it automatically. 

Once you’ve switched to the Pyrefly type engine, you will see a Pyrefly icon at the bottom, which you can hover over to check the version being used.

Please note that the integration currently works for local interpreter configurations. Support for Docker, Docker Compose, WSL, SSH, and multi-module projects is planned for future releases.

Pyrefly vs. the built-in type engine

Now let’s look at how Pyrefly and the built-in type engine behave in a complex Python project. In this FastAPI example, multiple files are typed, but in this file, the variable ref is incorrectly typed, causing four errors. When using the built-in type engine, the IDE identifies that something is wrong, but it suggests running further analysis to fix the problem, which requires an extra step.

Using Pyrefly as the type engine, the IDE reports errors immediately and highlights where they originate. However, it is worth noting that, in our example, there are four errors, but Pyrefly picks up only three of them. It misses the one in self._storage[ref].

Download the latest version of PyCharm and try it out

Ready to experience a dramatic leap in Python development performance? The Pyrefly type engine in PyCharm 2026.1.2 delivers the next generation of type checking. Engineered in Rust for unparalleled speed, it resolves files in as little as 0.5–1 seconds, significantly faster than the built-in engine. If you maintain large, complex Python codebases and prioritize robust typing, this feature is essential, as it allows you to delegate analysis to a faster engine and receive immediate type-related diagnostics. Download the latest version of PyCharm (2026.1.2) to unlock superior efficiency, scalability, and code insight.

To streamline this experience, PyCharm 2026.1.1 introduced built-in support for uv workspaces, as well as those managed by Poetry and Hatch. This new functionality – currently in Beta – allows the IDE to automatically manage dependencies and environments across your entire workspace.

Intelligent workspace detection

When you open a workspace, PyCharm can now derive its entire structure and all its dependencies directly from your pyproject.toml files. This allows the IDE to understand relationships between projects deeply, significantly reducing the amount of configuration you have to do manually.

Because this is a fundamental change to how PyCharm handles your workspace, we’ve implemented it as an opt-in feature. Here is what you need to know about the transition:

• Opt-in dialog: When you open a project, PyCharm may suggest enabling automatic detection for uv workspaces and Poetry/Hatch setups. 

• Manual configuration: You can toggle workspace detection in Settings | Project Structure.

• Configuration note: If you previously manually edited settings in .idea files, those settings may be reset when you agree to the new model.

Managing workspaces and their projects

PyCharm now provides an integrated experience that handles the complexities of multi-package setups in uv workspaces automatically. When you open a uv workspace, the IDE identifies the individual projects and their interdependencies, ensuring the project structure is ready for you to work with.

Visualizing workspace dependencies

Once the workspace is loaded, you can verify how your projects relate to one another. PyCharm presents these dependencies in Settings | Project Dependencies.

These relationships are derived directly from your configuration and are shown as read-only in the UI. To make changes to the dependency graph, you can edit the pyproject.toml file manually – PyCharm will then update its internal model.

Automatic environment configuration

PyCharm prioritizes a zero-config approach to your Python SDK. When you open a .py or pyproject.toml file within a project, the IDE performs an immediate check.

If a compatible environment already exists on your system, PyCharm automatically configures it as the SDK for that project. If no environment is detected, a file-level notification will appear suggesting that you create a new uv environment and install the necessary dependencies for that project.

Maintaining environment consistency

Beyond the initial setup, PyCharm continuously monitors the health of your environment to ensure it stays in sync with your defined requirements. 

If a dependency is not defined in your pyproject.toml file but is imported in your code, PyCharm will trigger a warning with a Sync project quick-fix to resolve these discrepancies.

Import management

PyCharm also assists when you are actively writing code by identifying gaps in your project configuration.

If you import a package that isn’t present in the environment and is not yet listed in the project’s pyproject.toml, the IDE will detect the omission. A quick-fix will suggest adding the package to the environment and updating the corresponding .toml file simultaneously.

Transparency via the Python Process Output tool window

While PyCharm automates the backend execution of commands – such as uv sync –all-packages – it still remains fully transparent.

You can track all executed commands and their live output in the Python Process Output tool window. If synchronization fails for an environment, you can analyze the specific error logs to quickly identify the root cause.

Poetry and Hatch workspaces

The logic for Poetry and Hatch workspaces follows this exact same workflow. PyCharm detects projects via their pyproject.toml files and manages the environments with the same automated precision.

The only minor difference is in tool selection – the suggested environment tool is determined by what you have specified in your pyproject.toml. If no tool is specified, PyCharm will prioritize uv (if installed) or a standard virtual environment to get you up and running quickly.

Looking ahead

This Beta version of the functionality is just the beginning of our focus on supporting complex workspace structures. We are already working on expanding the UI to allow creating new projects, linking dependencies, and activating the terminal for specific projects.

As we refine these features, your feedback is our best guide – please share your thoughts or report any issues on our YouTrack issue tracker.

AI is changing how Python developers learn, build, and contribute to open source. At the same time, long-standing questions around community, sustainability, data workflows, and web development are becoming even more important.

Python Unplugged on PyTV, a free online conference hosted by JetBrains PyCharm, brought these conversations together in over seven hours of talks and discussions with developers, maintainers, educators, and tool builders from across the Python ecosystem.

Don’t have time to watch the full event? This blog post gives you a quick overview of what’s happening in Python today, based on talks from 13 Python experts – from AI-assisted development and open-source sustainability to modern data processing, Django, and community building.

Watch the recap video

Want to see the highlights from Python Unplugged on PyTV? Watch the full recap video below.

JetBrains’ Dr. Jodie Burchell, Data Scientist and Python Advocacy Team Lead; Cheuk Ting Ho, Data Scientist and Developer Advocate; and Will Vincent, Python Developer Advocate, discuss the key talking points from the day.

Need a quick overview? Here are the highlights

If you’d rather get the key takeaways in a written format, we’ve broken down the biggest insights from the day below. From the evolving role of AI to the importance of the Python community, these are the moments that stood out most from Python Unplugged on PyTV.

Highlight 1: Python goes beyond scripts and prototypes

Many developers first come to Python through a specific use case: automating tasks, building prototypes, learning data science, or experimenting with AI and machine learning. That accessibility is one of Python’s strengths, but it’s only the entry point.

In her session, AI Practitioners Are Only Getting Half the Goodness of Python, Deb Nicholson, Executive Director at the PSF, discussed how many AI and ML practitioners use Python mainly as a scripting or prototyping language. But Python is also used to build and maintain real-world software, supported by frameworks, data tools, testing workflows, packaging standards, and an active open-source community.

This broader context matters for learning, too. In his How to Learn Python session, Mark Smith, Head of Python Ecosystem at JetBrains, focused on what comes after the fundamentals: building real projects, reading other people’s code, and developing the habits needed to move past “tutorial hell.”

AI can help, but it shouldn’t replace hands-on practice. As Cheuk noted in the recap video, one useful tip from Mark’s talk was to turn off AI features while learning, so beginners still build the judgment needed to understand and improve the code they work with.

Highlight 2: The continuing role of community in Python

Python’s success has always been rooted in its community, and that remains as true as ever. Georgi Ker, Director and Fellow at the PSF; Una Galyeva, Head of AI at Geobear Global; and Jessica Greene, Senior ML Engineer at Ecosia, showcased this in their How PyLadies Is Shaping the Future of Python discussion.

PyLadies is an international mentorship group focused on helping more women become active participants and leaders in the Python community. The success of initiatives like PyLadies highlights how inclusive spaces can broaden participation and shape the future of the language.

As Will noted in our recap video, “Being part of the community is not just the code. It’s the conferences, it’s the people, it’s the live events – that’s what makes Python special.”

Python depends on a culture of shared responsibility, and contributors play a vital role. As AI brings more people into the ecosystem, preserving these values becomes even more important. Travis Oliphant, creator of NumPy, touched on this in his insightful session, Community is More Than Code: People Are What Make Python Thrive, and Why That Will Continue in an AI-Enabled Era.

There’s also a strong link between community and innovation, as Carol Willing, Core Developer at JupyterLab, explained in her session, Conversation, Computation, and Community: Key Principles for Solving Scientific Problems With Jupyter Notebooks and AI Tools. Tools like Jupyter have thrived in part because they enable conversation, collaboration, and knowledge sharing among people.

Highlight 3: AI poses both a threat and an opportunity for Python open source

AI is fundamentally changing how developers interact with open source.

On the positive side, AI coding tools lower the barrier to entry and allow more people to contribute. However, this increased accessibility comes with trade-offs. Maintainers are now dealing with a higher volume of contributions, many of which require significant review or refinement. Deb Nicholson, Executive Director at the PSF, discussed this trade-off in more detail in her session, AI Practitioners Are Only Getting Half the Goodness of Python.

This shift places additional pressure on those responsible for maintaining open-source projects. While AI can accelerate development, it also risks introducing poorly structured or low-quality code at scale.

Paul Everitt, Developer Advocate at JetBrains; Georgi Ker, Director and Fellow at the PSF; and Carol Willing, Core Developer at JupyterLab, pondered this in their Open Source in the Age of Coding Agents discussion. Ultimately, AI can’t replace the human systems that sustain open source. Trust, collaboration, and shared ownership remain essential, and arguably become even more important as contribution volumes increase. The real challenge lies in ensuring communities remain healthy and resilient as they scale.

Highlight 4: AI has also revolutionized how Python practitioners work

Beyond its impact on open source, AI is transforming day-to-day development workflows.

As Marlene Mhangami, Senior Developer Advocate at Microsoft Agentic, explained in her A Practical Guide to Agentic Coding session, coding is emerging as a new paradigm in which developers delegate tasks to AI systems capable of planning, executing, and refining code. This means the developer’s role is moving toward orchestration and validation, requiring new skills in guiding and evaluating AI outputs.

At the same time, development is becoming more conversational and exploratory. In environments like Jupyter, AI tools help users iterate faster, test ideas more easily, and move more fluidly between thinking and coding.

AI is also having a tangible impact on frameworks like Django, as discussed by Sheena O’Connell, Board Member at the PSF, in her talk, Powering Up Django Development With Claude Code. AI tools can speed up development in Django by handling repetitive tasks such as boilerplate generation and debugging. However, this comes with a caveat – developers must remain critical and treat AI as a collaborator, not a source of truth.

For beginners, AI can be a powerful learning aid, but over-reliance can limit deeper understanding. Building projects, reading code, and actively solving problems remain essential for developing real expertise.

Highlight 5: The importance of open-source AI

The open-source AI ecosystem is expanding rapidly, bringing with it a growing landscape of models, datasets, and tools.

This openness drives collaboration, transparency, and innovation, making it easier for developers to experiment and build on existing work. At the same time, it introduces challenges around fragmentation and long-term sustainability.

As Merve Noyan, ML Engineer at Hugging Face, explained in her Open-Source AI Ecosystem session, platforms like Hugging Face play a key role in organizing this ecosystem and making it more accessible, while Python continues to connect tools, communities, and technologies.

Highlight 6: Context is key for effective AI agents

As AI systems become more advanced, the way they interact with their input data is becoming increasingly important. Tuana Çelik, Developer Relations Engineer at LlamaIndex, covered this in detail in her insightful Orchestrating Document-Centric Agents With LlamaIndex talk.

LlamaIndex enables developers to build document-centric AI agents that retrieve, index, and reason over large collections of information. By structuring how documents are ingested and queried, it provides the LLM with much more context for the text it is processing, helping produce more accurate, context-aware responses.

This is particularly valuable in knowledge bases and enterprise assistants, where understanding relationships between pieces of information is as important as accessing the data itself.

Highlight 7: How Polars is refining high-performance data processing

Polars is pushing Python data processing toward a more scalable, production-ready future, as Polars creator Ritchie Vink explained in his Towards Query Profiling in Polars session.

Its high-performance, lazy execution model allows queries to be optimized automatically behind the scenes. However, this level of abstraction can make it harder for developers to fully understand performance.

To address this, there’s a growing need for better tooling, particularly around query profiling. By exposing execution plans, memory usage, and bottlenecks, developers can make informed decisions and build more efficient data workflows.

With features like streaming execution, Polars is helping bridge the gap between local data processing and large-scale systems.

As Jodie highlighted in the recap discussion, this shift is bringing more advanced data concepts into everyday Python workflows. She commented, “It’s really interesting to see more big data ideas coming to local Python data processing.”

Highlight 8: The power of typing in modern Python

Typing in Python continues to evolve, with a growing focus on flexibility rather than rigid enforcement. Open-source Django projects creator Carlton Gibson shed more light on this during his talk, Static Islands, Dynamic Sea: Some Thoughts on Incremental Typing.

The talk highlighted how developers are increasingly adopting an incremental approach. By creating “static islands” within a dynamic codebase, they can improve reliability, maintainability, and tooling without sacrificing Python’s core strengths.

In our recap video, Will agreed with this sentiment, adding, “It doesn’t have to be all-or-nothing. We don’t have to turn Python into something that it’s not.”

This approach is particularly useful in large frameworks like Django, where typing can help define clearer boundaries while still preserving developer ergonomics.

Highlight 9: The Django renaissance: Debunking aging myths

Django remains a modern, actively developed framework, as Django Fellow Sarah Boyce revealed in her session, Django Has a Marketing Problem: Debunking the Myths That Won’t Die.

Many of the criticisms that it’s outdated or unscalable don’t reflect the current reality. In practice, Django continues to evolve and power a wide range of applications.

The challenge is less about Django’s capabilities and more about perception, as the Django community was called to champion its strengths, ongoing evolution, and real-world impact.

Shifting this narrative will be key to ensuring its continued relevance and adoption in the years ahead.

What’s next for Python Unplugged on PyTV?

Python Unplugged on PyTV was our first step in reimagining what a fully online community conference can look like, and the response was incredible.

Looking at the numbers, more than 5,500 people joined us during the livestream. Since then, we’ve had a further 110,000 watch the event recording, showing just how global and engaged the Python community really is.

We’d love to bring Python Unplugged on PyTV back next year. What would you like to see more of? Who should we invite as speakers? Are there topics we didn’t cover that you’d love to explore?

Drop your suggestions in the comments and help shape the future of Python Unplugged on PyTV.