[jboss-jira] [JBoss JIRA] (WFCORE-2301) Mount point not found exception raised by createTempFileWithAttributes on overlayfs [JDK-8165852]
Brian Stansberry (JIRA)
issues at jboss.org
Thu Feb 16 12:24:00 EST 2017
[ https://issues.jboss.org/browse/WFCORE-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13364952#comment-13364952 ]
Brian Stansberry edited comment on WFCORE-2301 at 2/16/17 12:23 PM:
--------------------------------------------------------------------
I think https://github.com/wildfly/wildfly-core/pull/2188 should work around this by dealing with the failure in Files.getFileStore(Path). We call this in four places in WildFly. This fixes two, the ones involved in your report. Lucene calls it as well in some util, but catches exceptions. The fourth is a management operation handler for an operation CLI/console users can use to learn about space available on the FS (PathInfoHandler). I left that as is, since no one is required to invoke that op and if it fails it seems reasonable that the user sees that.
was (Author: brian.stansberry):
I think https://github.com/wildfly/wildfly-core/pull/2188 should work around this by dealing with the failure in Files.getFileStore(Path). We call this in four places places in WildFly. This fixes two, the ones involved in your report. Lucene calls it as well in some util, but catches exceptions. The fourth is a management operation handler for an operation CLI/console users can use to learn about space available on the FS (PathInfoHandler). I left that as is, since no one is required to invoke that op and if it fails it seems reasonable that the user sees that.
> Mount point not found exception raised by createTempFileWithAttributes on overlayfs [JDK-8165852]
> -------------------------------------------------------------------------------------------------
>
> Key: WFCORE-2301
> URL: https://issues.jboss.org/browse/WFCORE-2301
> Project: WildFly Core
> Issue Type: Bug
> Components: Domain Management
> Environment: WildFly via KeyCloak 2.5.1.Final
> {code:xml}
> <eap.version>7.0.0.Beta</eap.version>
> <jboss.as.version>7.2.0.Final</jboss.as.version>
> <wildfly.version>10.0.0.Final</wildfly.version>
> {code}
> on Docker with overlayfs or overlayfs2 as storage driver
> \# docker info | grep -i storage
> aufs: works (e.g., boot2docker, legacy minikube)
> overlay (e.g., CoreOS, current minikube): problem
> devicemapper (e.g., CentOS): works
> overlay2 (e.g., Docker for Mac): problem
> Reporter: Bjoern Stuetz
> Assignee: Brian Stansberry
>
> Mount point not found exception raised by createTempFileWithAttributes on overlayfs [JDK-8165852], i.e.,
> /opt/jboss/bin/jboss-cli.sh --file=/opt/jboss/jboss-config.cli
> inside a Docker container running on overlayfs as storage driver
> causes (full stack trace below):
> {code:java}
> java.io.IOException: Mount point not foundImage
> at sun.nio.fs.LinuxFileStore.findMountEntry(LinuxFileStore.java:91)
> {code}
> triggered by
> {code:java}
> at org.jboss.as.controller.persistence.FilePersistenceUtils.createTempFileWithAttributes(FilePersistenceUtils.java:117)
> at org.jboss.as.controller.persistence.FilePersistenceUtils.writeToTempFile(FilePersistenceUtils.java:104)
> {code}
> due to OpenJDK bug/overlayfs bug.
> We acknowledge that this is in fact an OpenJDK AND/OR overlayfs bug. However everything seems to run fine in WildFly except once the backup of the config is triggered, for example by using the cli. Hence WildFly is of limited functionality when the more and more popular overlayfs storage driver is used, and the WildFly team might be interested in providing a workaround on their side since there is no indication the OpenJDK bug will be promptly fixed. We are happy to help in any way, we are still trying to find a workaround on the Java or WildFly side; but we might need insights on why findMountEntry is invoked.
> Full Stack Trace:
> {code:java}
> java.io.IOException: Mount point not foundImage
> at sun.nio.fs.LinuxFileStore.findMountEntry(LinuxFileStore.java:91)
> at sun.nio.fs.UnixFileStore.<init>(UnixFileStore.java:65)
> at sun.nio.fs.LinuxFileStore.<init>(LinuxFileStore.java:44)
> at sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:51)
> at sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:39)
> at sun.nio.fs.UnixFileSystemProvider.getFileStore(UnixFileSystemProvider.java:368)
> at java.nio.file.Files.getFileStore(Files.java:1461)
> at org.jboss.as.controller.persistence.FilePersistenceUtils.getPosixAttributes(FilePersistenceUtils.java:129)
> at org.jboss.as.controller.persistence.FilePersistenceUtils.createTempFileWithAttributes(FilePersistenceUtils.java:117)
> at org.jboss.as.controller.persistence.FilePersistenceUtils.writeToTempFile(FilePersistenceUtils.java:104)
> at org.jboss.as.controller.persistence.ConfigurationFilePersistenceResource.doCommit(ConfigurationFilePersistenceResource.java:55)
> at org.jboss.as.controller.persistence.AbstractFilePersistenceResource.commit(AbstractFilePersistenceResource.java:58)
> at org.jboss.as.controller.ModelControllerImpl$4.commit(ModelControllerImpl.java:781)
> at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:743)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:204)
> at org.jboss.as.controller.ModelControllerImpl$3.execute(ModelControllerImpl.java:659)
> at org.jboss.as.controller.ModelControllerImpl$3.execute(ModelControllerImpl.java:649)
> at org.jboss.as.controller.client.helpers.DelegatingModelControllerClient.execute(DelegatingModelControllerClient.java:63)
> at org.jboss.as.cli.embedded.ThreadContextsModelControllerClient.execute(ThreadContextsModelControllerClient.java:59)
> at org.jboss.as.cli.handlers.batch.BatchRunHandler.doHandle(BatchRunHandler.java:91)
> at org.jboss.as.cli.handlers.CommandHandlerWithHelp.handle(CommandHandlerWithHelp.java:88)
> at org.jboss.as.cli.impl.CommandContextImpl.handle(CommandContextImpl.java:776)
> at org.jboss.as.cli.impl.CommandContextImpl.handleSafe(CommandContextImpl.java:799)
> at org.jboss.as.cli.impl.CliLauncher.processFile(CliLauncher.java:334)
> at org.jboss.as.cli.impl.CliLauncher.main(CliLauncher.java:262)
> at org.jboss.as.cli.CommandLineMain.main(CommandLineMain.java:45)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jboss.modules.Module.run(Module.java:329)
> at org.jboss.modules.Main.main(Main.java:507)
> {code}
> Java Bug Overview:
> https://bugs.openjdk.java.net/browse/JDK-8165852
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/sun/nio/fs/LinuxFileStore.java#91
> Wildfly Stack Overflow issue, not solved:
> https://stackoverflow.com/questions/41022393/mount-point-not-found
> Background Info:
> http://mail.openjdk.java.net/pipermail/nio-dev/2016-October/003915.html
> A) chroot environment [1]
> B) Docker container with overlay and overlay2 storage drivers [2]
> C) btrfs file system with an unmounted sub-volume [2]
> [1] https://bugs.openjdk.java.net/browse/JDK-8165323 - cannot get FileStore in chroot environment
> [2] https://bugs.openjdk.java.net/browse/JDK-8165852 - cannot get FileStore for a file in overlayfs in Docker
> Docker file system/storage driver:
> https://docs.docker.com/engine/userguide/storagedriver/selectadriver/)
> Yum yum-plugin-ovl, similar problem:
> https://github.com/CentOS/sig-cloud-instance-images/issues/15
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
More information about the jboss-jira
mailing list