In the pursuit of enhancing Autogen’s capabilities, I drew inspiration from 0xlws’ fork supporting JavaScript. This led me to embark on a journey to modify Autogen, enabling robust support for Java code execution.


Setting up

Begin by ensuring that Java is installed on your Windows Subsystem for Linux (WSL) using the following command:

sudo apt install openjdk-17-jdk-headless

Modifying Autogen

Clone the latest changes from Autogen’s repository:

git clone https://github.com/microsoft/autogen.git

Key modifications were made to code_utils.py and conversable_agent.py to seamlessly integrate Java support. Here’s a snippet of the changes:

# conversable_agent.py
# @@ -1088,6 +1088,19 @@ class ConversableAgent(Agent):
            elif lang in ["java"]:
                if code.startswith("// filename: "):
                    filename = code[12 : code.find("\n")].strip()
                else:
                    filename = None
                exitcode, logs, image = self.run_code(
                    code,
                    lang="java",
                    filename=filename,
                    **self._code_execution_config,
                )
            elif lang == 'unknown':
                exitcode, logs, image = (0, "Language is unknown.", None)

# code_utils.py
# @@ -83,7 +83,8 @@ def infer_lang(code):
def infer_lang(code):
    """infer the language for the code.
    TODO: make it robust.
    """
    valid_prefixes = ["python ", "pip", "python3 ", "java ", "javac "]
    if any(code.startswith(prefix) for prefix in valid_prefixes):
        return "sh"

# @@ -215,7 +216,7 @@ def timeout_handler(signum, frame):
def _cmd(lang):
    if lang.startswith("python") or lang in ["bash", "sh", "powershell", "java"]:
        return lang
    if lang in ["shell"]:
        return "sh"
    if lang in ["ps1"]:
        return "powershell"
    raise NotImplementedError(f"{lang} not recognized in code execution")

For detailed changes, refer to the full git diff output.

autogen-code-modification-to-support-java

Recompile the modified changes:

pip install .

Generating Java code

To incorporate Java code execution, additional instructions have been added to the SYSTEM_MESSAGE:

For Java-specific code:
- If you want the user to save the code in a file before executing it, put // filename: <filename> inside the code block as the first line.
- Ensure that the filename matches the class name and has the same capitalization.

Here’s the entire app.py showcasing the integration:

import autogen

config_list = [
    {
        'api_key': 'NULL',
        'base_url': 'http://192.168.68.114:1234/v1'
    },
]

llm_config = {
    "cache_seed": 42,
    "temperature": 0,
    "config_list": config_list
}


SYSTEM_MESSAGE = """You are a helpful AI assistant.
Solve tasks using your coding and language skills.
For Java-specific code:
- If you want the user to save the code in a file before executing it, put // filename: <filename> inside the code block as the first line.
- Ensure that the filename matches the class name and has the same capitalization.
In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute.
    1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself.
    2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly.
Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill.
When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user.
If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user.
If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.
When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.
Reply "TERMINATE" in the end when everything is done.
"""

assistant = autogen.AssistantAgent(
    name="assistant",
    system_message=SYSTEM_MESSAGE,
    llm_config=llm_config,
    is_termination_msg = lambda x: "TERMINATE" in x.get("content", ""),
)

user_proxy = autogen.UserProxyAgent(
    name="user_proxy", 
    human_input_mode="NEVER",
    max_consecutive_auto_reply=3,
    llm_config=llm_config,
    code_execution_config={
        "work_dir": "coding",
        "use_docker": "openjdk"
    },
    system_message=""""Reply TERMINATE if the task has been solved at full satisfaction or the code executed without issue.
Otherwise, reply CONTINUE, or the reason why the task is not solved yet."""
)

user_proxy.initiate_chat(assistant, message="Write a Hello World application in java")

Similar to my previous post, I am using the latest version of autogen (requirements.txt: pyautogen==0.2.1) and the Dolphin 2.2.1 Mistral 7B model.

This is the sample run result:

autogen-hello-world-in-java


Extending Functionality to Additional Test Cases

Discover the versatility of Autogen by effortlessly switching between Java and Python code execution based on your specific requirements.

Switching to Python Requirements

Witness Autogen seamlessly transition to Python code execution:

user_proxy.initiate_chat(assistant, message="Write a Hello World application in python")

autogen-hello-world-in-python

Handling Fibonacci Sequences in Java

Explore Autogen’s flexibility in generating a Fibonacci sequence using Java:

user_proxy.initiate_chat(assistant, message="Calculate the first 10 terms of a Fibonacci sequence. Write the java code to generate and print these terms.")

autogen-fibonacci-sequence-in-java


Docker Integration for Java Code Execution

Ensure a streamlined execution process by utilizing Docker for Java code execution. The following steps were taken to optimize Autogen for Docker integration:

1. Removed openjdk from the WSL environment:

sudo apt remove openjdk-17-jdk-headless

2. Modified Autogen to support Java code execution within Docker

# code_utils.py
# @@ -348,13 +349,20 @@ def execute_code(
    # Prepare image_list based on language and use_docker preference
    if use_docker is True:
        lang = lang.lower()
        if lang in ["java"]:
            image_list = ["openjdk:22-slim-bulleye"]
        elif lang in ["python", "py"]:
            image_list = ["python:3-alpine", "python:3", "python:3-windowsservercore"]
        else:
            # Add more languages or a default image list if needed
            image_list = []
    elif isinstance(use_docker, str):
        image_list = [use_docker]
    else:
        image_list = use_docker

# @@ -399,7 +407,8 @@ def execute_code(
    # tag image based on the lang
    container.commit(repository=lang, tag=tag)

View the complete changes in the git diff output:

autogen-code-modification-to-support-java-in-docker

3. Configure Docker usage in the code_execution_config

    code_execution_config={
        "work_dir": "coding",
        "use_docker": "openjdk"
    },

4. Ensure proper Docker integration in the user_proxy configuration

user_proxy = autogen.UserProxyAgent(
    name="user_proxy", 
    human_input_mode="NEVER",
    max_consecutive_auto_reply=3,
    llm_config=llm_config,
    code_execution_config={
        "work_dir": "coding",
        "use_docker": "openjdk"
    },
    system_message=""""Reply TERMINATE if the task has been solved at full satisfaction or the code executed without issue.
Otherwise, reply CONTINUE, or the reason why the task is not solved yet."""
)

This is the result of the docker image ls:

autogen-java-code-execution-in-docker


Conclusion: Showcasing Java Code Execution in Autogen

This showcase exemplifies Autogen’s capabilities in executing Java codes seamlessly. The integration of Docker further optimizes the workflow, providing a robust development experience.