Hacking OpenJFX, Windows AppBar support

I like widget bars (if only because they tend to provide an analogue clock that I consult occasionally),
so over time I've used AB5k (Glossitope), WidgetFX, then Google Desktop, when the later was discontinued I thought I should do my own, so that it would only be discontinued when I'd stop maintaining it myself.
Eventually I started working on one using JavaFX.
Since I could not find a 'proper' way to have a side bar, I tried with this project to add some AppBar (doc and Petzold's article on the subject) support to OpenJFX.
It currently supports adding a main window that is an AppBar on any of the 4 edges, resizing and transparency.


To try it:

git clone https://github.com/lhochet/openjdk-jfx.git
git checkout appbar-12

then build and create a combined "JDK" with OpenJDK 11 or 12 (see that post)

then run:

public class JavaFXApplication1 extends Application {
    @Override
    public void start(final Stage primaryStage) {
        Button btn = new Button();
        btn.setText("Say 'Hello World'");
        btn.setOnAction(e -> {
            System.out.println("Hello World!");
            primaryStage.close();
        });
        VBox root = new VBox();
        root.getChildren().add(btn);
        Scene scene = new Scene(root, 250, 250); // only the dimension not related to the edge is used
        primaryStage.setTitle("Hello World!");
        primaryStage.setScene(scene);
//        primaryStage.initStyle(StageStyle.APPBAR_RIGHT);
//        primaryStage.initStyle(StageStyle.APPBAR_TOP);
//        primaryStage.initStyle(StageStyle.APPBAR_LEFT);
//        primaryStage.initStyle(StageStyle.APPBAR_BOTTOM);
        primaryStage.initStyle(StageStyle.APPBAR_RIGHT_TRANSPARENT);
//        primaryStage.initStyle(StageStyle.APPBAR_TOP_TRANSPARENT);
//        primaryStage.initStyle(StageStyle.APPBAR_LEFT_TRANSPARENT);
//        primaryStage.initStyle(StageStyle.APPBAR_BOTTOM_TRANSPARENT);
        primaryStage.show();
    }

    public static void main(String[] args) {
        launch(args);
    }
}

About the changes:
They mostly add StageStyles, hack transparency checks, and add the real AppBar support to GlassWindow.cpp/.h

• modules/javafx.graphics/src/main/java/com/sun/glass/ui/Window.java

Added styleMask constants to pass via JNI

• modules/javafx.graphics/src/main/java/com/sun/javafx/tk/quantum/GlassScene.java

Added the AppBar styles constants to the existing test to handle transparency

• modules/javafx.graphics/src/main/java/com/sun/javafx/tk/quantum/WindowStage.java

Defines a transparent window as being StageStyle.TRANSPARENT or a StageStyle.APPBAR_??_TRANSPARENT style.
It also initialise the Glass Windows implementation,
APPBAR_?? styles initialise a decorated window (titled, minimisable, maximisable, resizable), as well as the part of the mask for the AppBar edge,
APPBAR_??_TRANSPARENT styles initialises a non decorated transparent, minimisable, maximisable, resizable window,
as well as the part of the mask for the AppBar edge.

• modules/javafx.graphics/src/main/java/javafx/scene/Scene.java

Added the AppBar styles constants to the existing test to handle transparency

• modules/javafx.graphics/src/main/java/javafx/stage/Stage.java

Added the AppBar styles constants to the existing test to handle transparency

• modules/javafx.graphics/src/main/java/javafx/stage/StageStyle.java

Added the AppBar styles constants, one for each edge and transparent variant

• modules/javafx.graphics/src/main/native-glass/win/GlassWindow.h

Added AppBar related members to the GlassWindow class

• modules/javafx.graphics/src/main/native-glass/win/GlassWindow.cpp

The core of AppBar support.

includes Shellapi.h for the AppBar API signatures

• AppBar functions:

AppBarSetDefaultSize set the default size for the borderAppBarEdgeFor convert the given edge into its AppBar API constant
AppBarActivate activate the AppBar window
AppBarCreate create the new AppBar (reserve the visual space with the windows shell)
AppBarQuerySetPos query (check and adjust) the wanted AppBar position, then set it with the windows shell
AppBarCallback call back function for AppBar messages (copied from here) adjusted ABN_POSCHANGED to ensure it stays on the expected edge

• GlassWindow methods:

::GlassWindow initialise m_isAppBar and m_appBarBorder,
::Create initialise the window bounds based on the default (AppBarSetDefaultSize)
char *StringForMsg(UINT msg) added WM_LH_APPBAR_CALLBACK
::WindowProc handle WM_ENTERSIZEMOVE, WM_EXITSIZEMOVE to resize the AppBar space, WM_NCHITTEST to determine if the resize can occur (calls LHHandleHitTest), WM_LH_APPBAR_CALLBACK handle the AppBar call back messages (calls AppBarCallback)
::HandleCloseEvent and ::HandleDestroyEvent() removes the AppBar
::LHHandleHitTest only return a window edge if it is the resizable edge for the AppBar, returns HTCLIENT otherwise

• JNI functions:

Java_com_sun_glass_ui_win_WinWindow__1createWindow detect if it a flag for AppBar is passed then adjust the creation of the window accordingly
Java_com_sun_glass_ui_win_WinWindow__1createChildWindow adjusted to indicate that no AppBar edge was given
Java_com_sun_glass_ui_win_WinWindow__1setBounds override the window positioning to conform to the AppBar edge, also it is there that the AppBar is effectively created

(ping me on Twitter if I don't reply to comments)