Issue
I am writing a simple application and one of the features is the ability to serialize an object to an image so that the user can share a photo of their project with the build instructions embedded within it. Currently the other user can drag-and-drop the image into a JavaFX ListView and a listener will execute some code to decode and parse the embedded JSON.
This works fine if the image is stored locally, but I also want to allow users to drag and drop images from a browser without having to save them locally first. To test this I linked a working image in a basic HTML file so I could render it in Chrome.
Originally I was using the path to the file taken from the Dragboard, but when the image is coming from a browser I need (I think) to be able to accept the image directly, hence the overloaded decode() method below.
The problem I am having is that I am ending up with two slightly different byte arrays depending on whether the image comes from a browser or from somewhere in my main local storage. I don't know enough about images within Java (or in general) to understand why this is and haven't been able to find answers elsewhere. The browser sourced drag-drop produces a different enough byte array that I can't decode the message 100% properly, and therefore cannot deserialize it to an object. However, if I right click and save the browser based image, it loads correctly.
I have included a reproducible snippet and a test image below. My JDK is the most recent version of Amazon Corretto 8.
Reproducible snippet:
import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.ListView;
import javafx.scene.image.Image;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelReader;
import javafx.scene.image.WritablePixelFormat;
import javafx.scene.input.TransferMode;
import javafx.scene.layout.GridPane;
import javafx.stage.Stage;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferByte;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.nio.ByteBuffer;
import java.util.List;
public class MainApp extends Application {
private static final int OFFSET = 40;
private ListView<String> listView;
private GridPane gridPane;
@Override
public void start(Stage primaryStage) throws Exception {
listView = new ListView<>();
gridPane = new GridPane();
gridPane.addRow(0, listView);
setDragDropAction();
Scene scene = new Scene(gridPane, 250, 250);
primaryStage.setScene(scene);
primaryStage.sizeToScene();
primaryStage.show();
}
// the same drag-drop logic I am using in my project
private void setDragDropAction() {
gridPane.setOnDragOver(event -> {
if (event.getDragboard().hasFiles() || event.getDragboard().hasImage()) {
event.acceptTransferModes(TransferMode.ANY);
}
event.consume();
});
gridPane.setOnDragDropped(event -> {
List<File> files = event.getDragboard().getFiles();
try {
if (!files.isEmpty()) {
String decoded = decode(files.get(0));
System.out.println(decoded);
} else if (event.getDragboard().hasImage()) {
Image image = event.getDragboard().getImage();
String decoded = decode(image);
System.out.println(decoded);
}
} catch (IOException e) {
e.printStackTrace();
}
});
}
// decode using the file path
public String decode(File file) throws IOException {
byte[] byteImage = imageToByteArray(file);
return getStringFromBytes(byteImage);
}
// this results in a different byte array and a failed decode
public String decode(Image image) {
int width = (int) image.getWidth();
int height = (int) image.getHeight();
PixelReader reader = image.getPixelReader();
byte[] byteImage = new byte[width * height * 4];
WritablePixelFormat<ByteBuffer> format = PixelFormat.getByteBgraInstance();
reader.getPixels(0, 0, width, height, format, byteImage, 0, width * 4);
return getStringFromBytes(byteImage);
}
private String getStringFromBytes(byte[] byteImage) {
int offset = OFFSET;
byte[] byteLength = new byte[4];
System.arraycopy(byteImage, 1, byteLength, 0, (offset / 8) - 1);
int length = byteArrayToInt(byteLength);
byte[] result = new byte[length];
for (int b = 0; b < length; ++b) {
for (int i = 0; i < 8; ++i, ++offset) {
result[b] = (byte) ((result[b] << 1) | (byteImage[offset] & 1));
}
}
return new String(result);
}
private int byteArrayToInt(byte[] b) {
return b[3] & 0xFF
| (b[2] & 0xFF) << 8
| (b[1] & 0xFF) << 16
| (b[0] & 0xFF) << 24;
}
private byte[] imageToByteArray(File file) throws IOException {
BufferedImage image;
URL path = file.toURI().toURL();
image = ImageIO.read(path);
return ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
}
}
Another attempt at the decode method using SwingFXUtils.fromFXImage() which is also not working:
// Create a buffered image with the same imageType (6) as the type returned from ImageIO.read() with a local drag-drop
public String decode(Image image) throws IOException {
int width = (int) image.getWidth();
int height = (int) image.getHeight();
BufferedImage buffImageFinal = new BufferedImage(width, height, 6); // imageType = 6
BufferedImage buffImageTemp = SwingFXUtils.fromFXImage(image, null);
Graphics2D g = buffImageFinal.createGraphics();
g.drawImage(buffImageTemp, 0, 0, width, height, null);
g.dispose();
return getStringFromBytes(((DataBufferByte) buffImageFinal.getRaster().getDataBuffer()).getData());
}
Images showing the changes in the data:
Corner of image. Dissimilar pixels are highlighted in red.
Example image for testing:
The decoded testing message should print out ("This is a test message for my minimal reproducible example". Dragging the image from the browser does not work, saving the image locally and drag-dropping it does.
Solution
Based on a comment from the OP, event.getDragboard.getImage() loads an image with premultiplied alpha, while ImageIO.read() does not. One simple way to circumvent that is to get the url from the event (regardless of how the image is drag and dropped) and use ImageIO.read().
gridPane.setOnDragDropped(event -> {
if (event.getDragboard().hasUrl()) {
try {
URL path = new URL(event.getDragboard().getUrl());
String decoded = decode(path);
System.out.println(decoded);
} catch (IOException e) {
e.printStackTrace();
}
}
});
With the following modifications
public String decode(URL url) throws IOException {
byte[] byteImage = imageToByteArray(url);
return getStringFromBytes(byteImage);
}
private byte[] imageToByteArray(URL url) throws IOException {
BufferedImage image = ImageIO.read(url);
return ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
}
Note that loading from an url supports only a limited set of formats, as seen here.
Answered By - Reti43
