Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs on profiling with VisualVM #2251

Merged
merged 9 commits into from
Feb 18, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions source/docs/software/advanced-gradlerio/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ GradleRIO is the mechanism that powers the deployment of robot code to the roboR
code-formatting
gradlew-tasks
deploy-git-data
profiling-with-visualvm
Copy link
Collaborator

@sciencewhiz sciencewhiz Apr 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that advanced-gradlerio is the place for this. It does require modifying build.gradle. There isn't a great place that already exists, maybe telemetry?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not place it in telemetry, as I don't think we typically want to recommend users modify their build.gradle. If this is something common, we could enable it by default?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @Daltz333 that users should only be enabling this for diagnosis and shouldn't be in the build.gradle deployed for matches. I think that Software Tools might not be a bad place for this, but not Telemetry for the same reason as Daltz said.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I explained my concern with putting it in the advanced gradlerio section correctly. The other things in that section are permanent items that a team wants to change the build process. This is a temporary item to help with debugging. Because of that, I think it belongs elsewhere.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Software Tools?

176 changes: 176 additions & 0 deletions source/docs/software/advanced-gradlerio/profiling-with-visualvm.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
Profiling with VisualVM
=======================

This document is intended to familiarize the reader with the diagnostic tool that is `VisualVM <https://visualvm.github.io/>`__. VisualVM is a tool for profiling JVM based applications, such as viewing why an application is using a large amount of memory. This document assumes the reader is familiar with the *risks* associated with modifying their robot `build.gradle`. This tutorial also assumes that the user knows basic terminal/commandline knowledge.

Unpacking VisualVM
------------------

To begin, `download VisualVM <https://visualvm.github.io/download.html>`__ and unpack it to the WPILib installation folder. The folder is located at ``~/wpilib/`` where ``~`` indicates the users home directory. On Windows, this is ``C:\\users\\public\\wpilib``.

Setting up Gradle
-----------------

GradleRIO supports passing JVM launch arguments, and this is what is necessary to enable remote debugging. Remote debugging is a feature that allows a local machine (such as the user's desktop) to view important information about a remote target (in our case, a roboRIO). To begin, locate the ``frcJava`` code block located in the projects ``build.gradle``. Below is what is looks like.

.. code-block:: groovy

deploy {
targets {
roborio(getTargetTypeClass('RoboRIO')) {
// Team number is loaded either from the .wpilib/wpilib_preferences.json
// or from command line. If not found an exception will be thrown.
// You can use getTeamOrDefault(team) instead of getTeamNumber if you
// want to store a team number in this file.
team = project.frc.getTeamNumber()
debug = project.frc.getDebugOrDefault(false)

artifacts {
// First part is artifact name, 2nd is artifact type
// getTargetTypeClass is a shortcut to get the class type using a string

frcJava(getArtifactTypeClass('FRCJavaArtifact')) {
}

// Static files artifact
frcStaticFileDeploy(getArtifactTypeClass('FileTreeArtifact')) {
files = project.fileTree('src/main/deploy')
directory = '/home/lvuser/deploy'
}
}
}
}
}

We will be replacing

.. code-block:: groovy

frcJava(getArtifactTypeClass('FRCJavaArtifact')) {
}

with

.. code-block:: groovy

frcJava(getArtifactTypeClass('FRCJavaArtifact')) {
// Enable VisualVM connection
jvmArgs.add("-Dcom.sun.management.jmxremote=true")
jvmArgs.add("-Dcom.sun.management.jmxremote.port=1198")
jvmArgs.add("-Dcom.sun.management.jmxremote.local.only=false")
jvmArgs.add("-Dcom.sun.management.jmxremote.ssl=false")
jvmArgs.add("-Dcom.sun.management.jmxremote.authenticate=false")
jvmArgs.add("-Djava.rmi.server.hostname=10.XX.XX.2") // Replace XX.XX with team number
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it work with mDNS? If so, that would avoid the needing to specify different IPs for different networking configs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure, pending testing.

}

We are adding a few arguments here. In order:

* Enable remote debugging
* Set the remote debugging port to 1198
* Allow listening from remote targets
* Disable SSL authentication being required
* Set the hostname to the roboRIOs team number. Be sure to replace this.

.. important:: The hostname when connected via USB-B should be ``172.22.11.2``.

Running VisualVM
----------------

Launching VisualVM is done via the commandline with a few parameters. First, we navigate to the directory containing VisualVM. Then, launch it with parameters, passing it the WPILib JDK path. On a Windows machine, it looks like the following:

.. code-block:: bash

cd "C:\Users\Public\wpilib\visualvm_216\bin"
./visualvm --jdkhome "C:\Users\Public\wpilib\2023\jdk"

.. important:: The exact path ``visualvm_216`` may vary and depends on the version of VisualVM downloaded.

This should launch VisualVM. Once launched, open the :guilabel:`Add JMX Connection` dialog.

.. image:: images/visualvm/visualvm-addconn.png
:alt: Add visualvm connection menu option
:width: 700

Once opened, configure the connection details and ensure that :guilabel:`Do not require SSL connection` is ticked.

.. image:: images/visualvm/visualvm-dialog.png
:alt: VisualVM connection dialog is ticked
:width: 700

If correctly done, a new menu option in the left-hand sidebar will appear. Clicking on it will show you a detailed dashboard of the running JVM application.

.. image:: images/visualvm/visualvm-dash.png
:alt: VisualVM diagnostics dashboard
:width: 700

Analyzing Function Timings
--------------------------

An important feature of VisualVM is the ability to view how much time a specific function is taking up. This is *without* having a code debugger attached. To begin, click on the :guilabel:`Sampler` tab and then click on :guilabel:`CPU`. This will immediately give a breakdown of what functions are taking CPU time.

.. image:: images/visualvm/visualvm-function-profiling.png
:alt: Analyzing the VisualVM function timing tree
:width: 700

The above screenshot shows a breakdown of the total time a specific function takes. You can see that ``totallyNotSlowFunction()`` accounts for ``61.9%`` of the robot program CPU time. We can then correlate this to our robot program. In ``totallyNotSlowFunction()``, we have the following code.

.. code-block:: Java

public static void totallyNotSlowFunction() {
for (int i = 0; i < 2000; i++) {
System.out.println("HAHAHAHA");
}
}

In this code snippet, we can identify 2 major causes of concern. A long running ``for`` loop blocks the rest of the robot program from running. Additionally, ``System.out.println()`` calls on the roboRIO are typically quite expensive. We found this information by profiling the Java application on the roboRIO!

Creating a Heap Dump
--------------------

Besides viewing the remote systems CPU and memory usage, VisualVM is most useful by creating a **Heap Dump**. When a Java object is created, it resides in an area of memory called the heap. When the heap is full, a process called `garbage collection <https://www.geeksforgeeks.org/garbage-collection-java/>`__ begins. Garbage collection can be a common cause of loop overruns in a traditional Java robot program.

To begin, ensure you are on the :guilabel:`Monitor` tab and click :guilabel:`Heap Dump`.

.. image:: images/visualvm/visualvm-perform-heapdump.png
:alt: Location of heap dump button in VisualVM
:width: 700

This heap dump will be stored on the target system (roboRIO) and must be retrieved using SFTP. For this tutorial, `FileZilla <https://filezilla-project.org/>`__ will be the tool of choice. Open FileZilla and connect to the roboRIO with the following parameters

- Host: ``sftp://172.22.11.2`` or ``sftp://10.TE.AM.2``
- Username: ``lvuser``
- Password: Leave blank
- Port: Leave blank

If connected successfully, you'll see a list of files on the roboRIO.

.. image:: images/visualvm/filezilla-connection.png
:alt: Filezilla list of files
:width: 700

Navigate to the ``/tmp/`` directory with FileZilla and download it to your machine. Once downloaded, the dump can be analyzed with VisualVM.

Analyzing a Heap Dump
---------------------

Reopen VisualVM if closed using the previous instructions. Then click on :guilabel:`File` and :guilabel:`Load`. Navigate to the retrieved dump file and load it.

.. image:: images/visualvm/visualvm-viewing-dump.png
:alt: Viewing a dump in VisualVM
:width: 700

Clicking on :guilabel:`Summary` and selecting :guilabel:`Objects` instead will show a breakdown of objects by quantity. The below screenshot showcases a completely empty robot program, and then one that creates an million large ``ArrayList`` of integers.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to see where those million integers is created? I'm not sure there's much useful for teams based on what's shown in the screenshot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'll have to find that out myself

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In dotMemory, it shows the creation call stack, I wonder if VisualVM has something similar.


Blank robot program:

.. image:: images/visualvm/visualvm-objects1.png
:alt: List of objects in a blank robot program
:width: 700

with an ``ArrayList`` of ~10000 integers.

.. image:: images/visualvm/visualvm-objects2.png
:alt: List of objects in a modified robot program
:width: 700

For more information on VisualVM, check out the `VisualVM documentation pages <https://visualvm.github.io/documentation.html>`__.