[FIXED] JDK-18 foreign functions: How to work with multiple functions that have the same name, but come from different native libraries?

Issue

Let's say I loaded a native library gg3 that has a function get_gg(), and would like to load another native library gg4 that also has a get_gg() function - how can I look one up in a "library aware" manner (for example, use the get_gg() from gg4 on certain occasions, and otherwise from gg3)?

I tried CLinker.systemCLinker().lookup() but that seems to just fetch the first symbol that is found in the loaded native libraries.

After searching in the internal classes, I found SystemLookup.libLookup() which seemed promising but isn't public, so I implemented a similar method to lookup a loaded native lib:

private static SymbolLookup libLookup(Function<NativeLibraries, NativeLibrary> loader) {
        NativeLibrary lib = loader.apply(NativeLibraries.rawNativeLibraries(SystemLookup.class, false));
        return name -> {
            Objects.requireNonNull(name);
            try {
                long addr = lib.lookup(name);
                return addr == 0 ?
                        Optional.empty() :
                        Optional.of(NativeSymbol.ofAddress(name, MemoryAddress.ofLong(addr), ResourceScope.globalScope()));
            } catch (NoSuchMethodException e) {
                return Optional.empty();
            }
        };
    }

However, this apparently is prohibited, because I cannot load the library from another class loader (and I don't know what class loader was used originally):

java.lang.UnsatisfiedLinkError: Native Library already loaded in another classloader

So how can I either find & use the appropriate class loader, or manage to find the library without re-loading it, so I can look for a symbol specifically from it?

Solution

FWIW, the API needed to do this was made public in 19

Before that, the advice was to manually implement this by binding dlopen, dlclose, and dlsym (or the Windows equivalents) to load the library and look up symbols.

e.g. on my Windows machine:

public class Main {
    
    static final MethodHandle MH_LoadLibraryA;
    static final MethodHandle MH_FreeLibrary;
    static final MethodHandle MH_GetProcAddress;

    static {
        System.loadLibrary("Kernel32");
        SymbolLookup lookup = SymbolLookup.loaderLookup();
        CLinker linker = CLinker.systemCLinker();
        MH_LoadLibraryA = linker.downcallHandle(
            lookup.lookup("LoadLibraryA").orElseThrow(),
            FunctionDescriptor.of(ADDRESS, ADDRESS));
        MH_FreeLibrary = linker.downcallHandle(
            lookup.lookup("FreeLibrary").orElseThrow(),
            FunctionDescriptor.of(JAVA_INT, ADDRESS));
        MH_GetProcAddress = linker.downcallHandle(
            lookup.lookup("GetProcAddress").orElseThrow(),
            FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS));
    }

    static MemoryAddress LoadLibraryA(String name) {
        try (ResourceScope scope = ResourceScope.newConfinedScope()) {
            SegmentAllocator allocator = SegmentAllocator.nativeAllocator(scope);
            return (MemoryAddress) MH_LoadLibraryA.invokeExact((Addressable) allocator.allocateUtf8String(name));
        } catch (Throwable t) {
            throw new RuntimeException(t);
        }
    }

    static boolean FreeLibrary(MemoryAddress handle) {
        try {
            return ((int) MH_FreeLibrary.invokeExact((Addressable) handle)) != 0;
        } catch (Throwable t) {
            throw new RuntimeException(t);
        }
    }

    static MemoryAddress GetProcAddress(MemoryAddress handle, String name) {
        try (ResourceScope scope = ResourceScope.newConfinedScope()) {
            SegmentAllocator allocator = SegmentAllocator.nativeAllocator(scope);
            return (MemoryAddress) MH_GetProcAddress.invokeExact((Addressable) handle, (Addressable) allocator.allocateUtf8String(name));
        } catch (Throwable t) {
            throw new RuntimeException(t);
        }
    }
    
    static SymbolLookup libraryLookup(String libraryName, ResourceScope scope) {
        MemoryAddress handle = LoadLibraryA(libraryName);
        if (handle == MemoryAddress.NULL) {
            throw new IllegalArgumentException("Cannot find library: " + libraryName);
        }
        scope.addCloseAction(() -> FreeLibrary(handle));
        return name -> {
            var addr = GetProcAddress(handle, name);
            return addr == MemoryAddress.NULL ?
                Optional.empty() : Optional.of(NativeSymbol.ofAddress(name, addr, scope));
        };
    }

    public static void main(String[] args) throws Throwable {
        try (ResourceScope scope = ResourceScope.newConfinedScope()) {
            SymbolLookup lookup = libraryLookup("gg", scope);
            MethodHandle handle = CLinker.systemCLinker().downcallHandle(
                lookup.lookup("get_gg").orElseThrow(),
                FunctionDescriptor.of(JAVA_INT));
            int x = (int) handle.invoke();
            System.out.println(x);
        }
    }
}

Answered By - Jorn Vernee
Answer Checked By - David Goodson (JavaFixing Volunteer)