Exploring Unity Editor Handle Caps

Starting-OFF

This is going to be a relatively short write on adding and modifying custom handle caps in a Unity Editor. If you prefer to go through this tutorial as a video, you can check it out here on my channel.
Let’s first create an empty scene with a plane, a sphere, a camera and a directional light. We are going to add a script on the sphere to make it spawn our handle caps. Handle caps are basically those gizmos that you use to move, rotate and scale an object. Unity provides is with quite a few shapes to meet our requirements. So, let’s set up the initial scene and get started exploring Unity editor Handle caps.

unity handle caps initial scene
InitialScene

I created a “CustomHandle.cs” script that will be added to sphere and it’s corresponding script in “Editor/CustomHandleEditor.cs”. We will be working with these two files.
CustomHandle.cs is pretty simple and is self explanatory. We expose handleParams to the editor so that we can modify a few parameters of handle caps directly from editor.

///CustomHandle.cs///
public enum HandleTypes
{
    Arrow, Circle, Cone, Cube, Dot, Rectangle, Sphere
}

[System.Serializable]
public class HandleParams
{
    public float size = 1.0f;
    public float offset = 0.0f;
    public HandleTypes Type = HandleTypes.Arrow;
}

public class CustomHandle : MonoBehaviour
{
    [SerializeField]
    public HandleParams handleParams;
}

I’m breaking this down into 3 steps:
– Drawing handle caps.
– Highlighting them if mouse is closest to one of the handle caps we created.
– Making them respond to mouse events.

Drawing Handle Caps

Crux of our logic will stay inside OnSceneGUI() function in Editor script. To render the handle caps, we just need to call the appropriate function in Unity’s Handle class and pass right parameters. I have a helper function that lets me pass additional parameters. To draw anything, we are going to do it when editor fires a “Repaint” event. I assigned 11, 12, 13 as IDs for forward, right and up handle caps respectively. They are going to be useful in next steps.

///CustomHandleEditor.cs///
[CustomEditor(typeof(CustomHandle))]
public class CustomHandleEditor : Editor
{
    private void OnSceneGUI()
    {
        CustomHandle handle = target as CustomHandle;
if(Event.current.type == EventType.Repaint)
        {
            EventType eventType = EventType.Repaint;
            //Forward
            CreateHandle(11, handle.transform.position + handle.transform.forward * handle.handleParams.offset,
                       Quaternion.FromToRotation(Vector3.forward, handle.transform.forward),
                        handle.handleParams.size, handle.handleParams.Type, eventType);
            
            //Right
            CreateHandle(12, handle.transform.position + handle.transform.right * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.right), 
                        handle.handleParams.size, handle.handleParams.Type, eventType);

            //up
            CreateHandle(13, handle.transform.position + handle.transform.up * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.up), 
                        handle.handleParams.size, handle.handleParams.Type, eventType);
        }
    }
private void CreateHandle(int id, Vector3 position, Quaternion rotation, float size, HandleTypes handleTypes, EventType eventType )
    {
        Handles.ArrowHandleCap(id, position, rotation, size, eventType);
    }
}

Compiling and running the code above should draw handle caps in editor with customizable size and offset params as follows:

Unity Handle Caps Rendered
Handle caps rendered
Highlighting Them

Let’s now try and make them change color when mouse hovers over them. To do that, we need to create handle caps during “Layout” event and then re-draw them during a repaint event. This is because, we would be using “HandleUtility.nearestControl” to retrieve handle nearest to our mouse. To compare out handle ID with nearest handle ID and draw the handle with proper color, we would need this information prior to the “Repaint” event. This is how the code looks after making the necessary changes. We would basically be creating handle caps twice. I weirdly could not find any utility function to retrieve handles from handle IDs. If there was such a function, we would retrieve the Handle from ID and repaint.

///CustomHandleEditor.cs///
private void OnSceneGUI()
    {
        CustomHandle handle = target as CustomHandle;

        int hoverIndex = -1;

        hoverIndex = HandleUtility.nearestControl;
        if(Event.current.type == EventType.Layout)
        {
            EventType eventType = EventType.Layout;
            //Forward
            CreateHandle(11, handle.transform.position + handle.transform.forward * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.forward),
                        handle.handleParams.size, handle.handleParams.Type, eventType);
            
            //Right
            CreateHandle(12, handle.transform.position + handle.transform.right * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.right), 
                        handle.handleParams.size, handle.handleParams.Type, eventType);

            //up
            CreateHandle(13, handle.transform.position + handle.transform.up * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.up), 
                        handle.handleParams.size, handle.handleParams.Type, eventType);
        }

        //Highlight when mouse is near
        if(Event.current.type == EventType.Repaint)
        {
            EventType eventType = EventType.Repaint;
            //Forward
            Handles.color = hoverIndex == 11 ? Color.magenta : Handles.zAxisColor;
            CreateHandle(11, handle.transform.position + handle.transform.forward * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.forward),
                        handle.handleParams.size, handle.handleParams.Type, eventType);
            
            //Right
            Handles.color = hoverIndex == 12 ? Color.magenta : Handles.xAxisColor;
            CreateHandle(12, handle.transform.position + handle.transform.right * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.right), 
                        handle.handleParams.size, handle.handleParams.Type, eventType);

            //up
            Handles.color = hoverIndex == 13 ? Color.magenta : Handles.yAxisColor;
            CreateHandle(13, handle.transform.position + handle.transform.up * handle.handleParams.offset,
                        Quaternion.FromToRotation(Vector3.forward, handle.transform.up), 
                        handle.handleParams.size, handle.handleParams.Type, eventType);
        }
    }
Interaction

Finally, to make them interactable, I added 2 members to the editor class: a “Vector3 prevMousePosition” and an “int nearestHandle”. Here, we have mouse down, up and drag events and each sets the variables appropriately. Append this code to “OnSceneGUI” function that we’ve been modifying and you should have handle caps that you can click and drag around.

///CustomHandleEditor.cs///
//Interaction
        if(Event.current.type == EventType.MouseDown)
        {
            nearestHandle = HandleUtility.nearestControl;
            prevMousePosition = Event.current.mousePosition;
            Debug.Log("handle Down " + nearestHandle);
        }
        if(Event.current.type == EventType.MouseUp)
        {
            prevMousePosition = Vector3.zero;
            nearestHandle = -1;
        }
        if(Event.current.type == EventType.MouseDrag)
        {
            Debug.Log("Dragging " + nearestHandle);
            if(nearestHandle == 11)
            {
                float move = HandleUtility.CalcLineTranslation(prevMousePosition, Event.current.mousePosition,
                                                    handle.transform.position, handle.transform.forward);
                
                handle.transform.position += handle.transform.forward * move;
            }
            if(nearestHandle == 12)
            {
                float move = HandleUtility.CalcLineTranslation(prevMousePosition, Event.current.mousePosition,
                                                    handle.transform.position, handle.transform.right);

                handle.transform.position += handle.transform.right * move;
            }
            if(nearestHandle == 13)
            {
                float move = HandleUtility.CalcLineTranslation(prevMousePosition, Event.current.mousePosition,
                                                    handle.transform.position, handle.transform.up);

                handle.transform.position += handle.transform.up * move;
            }
            prevMousePosition = Event.current.mousePosition;
        }

That is all folks! I didn’t elaborate much in this post because most of the concepts are pretty straight forward and math is also quite simple. Please do post a comment if anything requires a deeper analysis.

,

One response to “Exploring Unity Editor Handle Caps”

Leave a Reply

Your email address will not be published. Required fields are marked *