Primate Labs, the creator of Geekbench, has launched Geekbench AI, a versatile AI benchmarking tool designed to assess the performance of AI workloads by using real-world machine learning tasks. This tool evaluates a device’s CPU, GPU, and NPU capabilities, ensuring they are ready for current and future machine learning applications.
Introduction to Geekbench AI 1.0
Geekbench AI provides a comprehensive suite for testing various machine learning, deep learning, and AI-specific workloads. The tool is designed to deliver consistent performance evaluations across multiple platforms, making it a valuable resource for software developers to maintain app performance, for hardware engineers to measure architectural improvements, and for users to troubleshoot device performance issues.
Supported Platforms
Geekbench AI is compatible with several platforms, including:
– Android: Android 12
– iOS: iOS 17
– Linux: Ubuntu 22.04 LTS
– macOS:macOS 14
– Windows:Windows 10
Initially known as “Geekbench ML,” the tool was rebranded as “Geekbench AI” to align more closely with industry terminology and clarify its intended purpose.
Key Features of Geekbench AI
1. Performance Scoring:
Geekbench AI delivers three overall scores—Single Precision, Half Precision, and Quantized. This scoring system reflects the diverse AI hardware designs found in different devices. By using a multi-dimensional scoring approach, Geekbench AI captures AI performance more accurately than a single metric.
2. Speed and Accuracy:
The benchmark includes a new accuracy measurement for each test, emphasizing the importance of both speed and accuracy in AI performance. This allows developers to evaluate the trade-offs between performance and efficiency, particularly when using smaller data types.
3. Frameworks and Datasets:
Geekbench AI 1.0 supports various frameworks, including OpenVINO and TensorFlow Lite delegates (Samsung ENN, ArmNN, Qualcomm QNN). It uses extensive datasets that closely resemble real-world AI applications, thereby improving the accuracy of evaluations.
4. Runtime:
Each AI workload in Geekbench AI is run for at least five iterations and a minimum of one second, ensuring that devices reach their peak performance during the testing phase.
Workloads in Geekbench AI
Geekbench AI covers a wide range of computer vision and natural language processing (NLP) tasks:
– Computer Vision:
– Image Classification: Utilizes MobileNetV1 to predict the category of objects in images.
-Image Segmentation:Uses DeepLabV3+ to classify each pixel within an image.
– Pose Estimation: Employs OpenPoseV2 to identify human body parts in an image.
– Object Detection:Leverages SSD to detect objects and their locations within an image.
– Face Detection: Uses Retinaface to detect and localize faces.
– Depth Estimation: Applies ConvNets to create a depth map of an image.
– Image Super Resolution: Enhances low-resolution images using RFDN.
– Style Transfer:Blends content and style images using a Fast Real-Time Style Transfer model.
–Natural Language Processing:
– Text Classification: Uses BERT-Tiny for sentiment analysis.
– Machine Translation: Employs Transformer architecture to translate text between languages.
Integration and Professional Licenses
Geekbench AI is integrated with the Geekbench Browser, which allows users to compare AI performance across different devices and access the latest benchmark results.
For professionals and corporate users, Geekbench AI Pro offers additional features, including automated testing tools, an offline mode, a commercial use license, and basic email support.
Availability
Geekbench AI 1.0 is available for download on various platforms, including Windows, macOS, Linux, and mobile platforms through the Google Play Store and Apple App Store. This wide availability makes it accessible to a broad audience, from individual users to large corporations, who need a reliable tool to measure AI performance across different devices and environments.
This new tool not only provides detailed insights into AI performance but also helps in optimizing and improving future AI developments by offering a standardized and comprehensive benchmarking solution.
