Java_Language

Java Tutorial2D Graphics in JavaJava Stack-Walking APIClass - Java ReflectionCommon Java PitfallsDocumenting Java CodeGenerating Java CodeInstalling Java (Standard Edition)Java AgentsJava Alternative CollectionsJava AnnotationsJava Apache Commons LangJava AppDynamics and TIBCO BusinessWorks Instrumentation for Easy IntegrationJava AppletsJava ArraysJava AssertingJava Atomic TypesJava AudioJava AutoboxingJava Basic Control StructuresJava BenchmarksJava BigDecimalJava BigIntegerJava Bit ManipulationJava BufferedWriterJava ByteBufferJava Bytecode ModificationJava C++ ComparisonJava Calendar and its SubclassesJava Character encodingJava Choosing CollectionsJava Classes and ObjectsJava ClassloadersJava Collection Factory MethodsJava CollectionsJava Command line Argument ProcessingJava Comparable and ComparatorJava Compiler - javacJava CompletableFutureJava Concurrent CollectionsJava Concurrent Programming (Threads)Java Console I/OJava ConstructorsJava Converting to and from StringsJava Creating Images ProgrammaticallyJava Currency and MoneyJava Date ClassJava Dates and Time (java.time.*)Java Default MethodsJava deploymentJava Dequeue InterfaceJava Disassembling and DecompilingJava Dynamic Method DispatchJava Editions, Versions, Releases and DistributionsJava EncapsulationJava Enum MapJava Enum starting with numberJava EnumsJava EnumSet classJava Exceptions and exception handlingJava Executor, ExecutorService and Thread poolsJava ExpressionsJava File I/OJava FileUpload to AWSJava Floating Point OperationsJava Fluent InterfaceJava FTP (File Transfer Protocol)Java Functional InterfacesJava GenericsJava Getters and SettersJava HashtableJava HttpURLConnectionJava Immutable ClassJava Immutable ObjectsJava InheritanceJava InputStreams and OutputStreamsJava InterfacesJava Iterator and IterableJava JavaBeanJava JAX-WSJava JAXBJava JMXJava JNDIJava JShellJava Just in Time (JIT) compilerJava JVM FlagsJava JVM Tool InterfaceJava Lambda ExpressionsJava LinkedHashMapJava List vs SET



Java Benchmarks

From WikiOD

Writing performance benchmarks in java is not as simple as getting System.currentTimeMillis() in the beginning and in the end and calculating the difference. To write valid performance benchmarks, one should use proper tools.

Simple JMH example[edit | edit source]

One of the tools for writing proper benchmark tests is JMH. Let's say we want to compare performance of searching an element in HashSet vs TreeSet.

The easiest way to get JHM into your project - is to use maven and shade plugin. Also you can see pom.xml from JHM examples.

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.0.0</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <finalName>/benchmarks</finalName>
                        <transformers>
                            <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>org.openjdk.jmh.Main</mainClass>
                            </transformer>
                        </transformers>
                        <filters>
                            <filter>
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>META-INF/*.SF</exclude>
                                    <exclude>META-INF/*.DSA</exclude>
                                    <exclude>META-INF/*.RSA</exclude>
                                </excludes>
                            </filter>
                        </filters>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

<dependencies>
    <dependency>
        <groupId>org.openjdk.jmh</groupId>
        <artifactId>jmh-core</artifactId>
        <version>1.18</version>
    </dependency>
    <dependency>
        <groupId>org.openjdk.jmh</groupId>
        <artifactId>jmh-generator-annprocess</artifactId>
        <version>1.18</version>
    </dependency>
</dependencies>

After this you need to write benchmark class itself:

package benchmark;

import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;

import java.util.HashSet;
import java.util.Random;
import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.TimeUnit;

@State(Scope.Thread)
public class CollectionFinderBenchmarkTest {
    private static final int SET_SIZE = 10000;

    private Set<String> hashSet;
    private Set<String> treeSet;

    private String stringToFind = "8888";

    @Setup
    public void setupCollections() {
        hashSet = new HashSet<>(SET_SIZE);
        treeSet = new TreeSet<>();

        for (int i = 0; i < SET_SIZE; i++) {
            final String value = String.valueOf(i);
            hashSet.add(value);
            treeSet.add(value);
        }

        stringToFind = String.valueOf(new Random().nextInt(SET_SIZE));
    }

    @Benchmark
    @BenchmarkMode(Mode.AverageTime)
    @OutputTimeUnit(TimeUnit.NANOSECONDS)
    public void testHashSet(Blackhole blackhole) {
        blackhole.consume(hashSet.contains(stringToFind));
    }

    @Benchmark
    @BenchmarkMode(Mode.AverageTime)
    @OutputTimeUnit(TimeUnit.NANOSECONDS)
    public void testTreeSet(Blackhole blackhole) {
        blackhole.consume(treeSet.contains(stringToFind));
    }
}

Please keep in mind this blackhole.consume(), we'll get back to it later. Also we need main class for running benchmark:

package benchmark;

import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

public class BenchmarkMain {
    public static void main(String[] args) throws RunnerException {
        final Options options = new OptionsBuilder()
                .include(CollectionFinderBenchmarkTest.class.getSimpleName())
                .forks(1)
                .build();

        new Runner(options).run();
    }
}

And we're all set. We just need to run mvn package (it will create benchmarks.jar in your /target folder) and run our benchmark test:

java -cp target/benchmarks.jar benchmark.BenchmarkMain

And after some warmup and calculation iterations, we will have our results:

# Run complete. Total time: 00:01:21

Benchmark                                  Mode  Cnt   Score    Error  Units
CollectionFinderBenchmarkTest.testHashSet  avgt   20   9.940 ±  0.270  ns/op
CollectionFinderBenchmarkTest.testTreeSet  avgt   20  98.858 ± 13.743  ns/op

About that blackhole.consume(). If your calculations do not change the state of your application, java will most likely just ignore it. So, in order to avoid it, you can either make your benchmark methods return some value, or use Blackhole object to consume it.

You can find more information about writing proper benchmarks in Aleksey Shipilëv's blog, in Jacob Jenkov's blog and in java-performance blog: 1, 2.

Credit:Stack_Overflow_Documentation