[wildfly-dev] Shall we limit size of the deployment in WildFly?
Brian Stansberry
brian.stansberry at redhat.com
Thu Dec 3 13:28:36 EST 2015
On 12/3/15 9:04 AM, Emmanuel Hugonnet wrote:
> Hi,
> Here is my ideas and questions for displaying the filesystem usage for the various paths:
> * create a handler like the ResolvePath handler so that it becomes really easy to add the metric to any path attribute.
Good idea.
> * adding it per default in the PathResourceDefinition to make it widely available 'for free'.
Yes, for sure.
> * which unit should we use to express it ? Is bytes ok ?
I think so, yes. You can't gain back lost precision.
Your variant of ResolvePathHandler could take an enum param if people
want some other unit.
Haha, I guess ^^^ means you can gain back lost precision, so now I have
no strong opinion on the default precision. :)
> * do we want to have all the values in the server-environment or are those under /path sufficient ?
I believe there are synthetic path resources for all of the paths from
server-environment.
> * do we want to aggregate those values in a 'single' metric and if so which paths should we select ?
IMHO no. KISS principle.
> * for space availability this might depend on the different mount points:
> - should we provide it for every path since we wouldn't know in advance to which mount point each path is on
I don't understand this question. How were you planning to get the space
availability information?
My instinct is the closer this conforms to what the standard java (n)io
API provides the better.
> - or maybe this info is not that important
>
> Cheers,
> Emmanuel
>
> Le 24/11/2015 08:43, Tristan Tarrant a écrit :
>> https://issues.jboss.org/browse/WFCORE-1159
>>
>> Tristan
>>
>> On 24/11/2015 00:51, Brian Stansberry wrote:
>>> A WFCORE JIRA for Tristan's specific request would be good.
>>>
>>> I don't think the discussion on this thread is close enough to a
>>> consensus on how to practically use this data in handling deployment ops
>>> for it to be useful to bring that aspect into a JIRA.
>>>
>>>
>>> On 11/23/15 4:13 AM, Lin Gao wrote:
>>>> >From what I learned of this thread, a service to expose the file system space availability
>>>> needs to be implemented first, then provides a way to check whether the server locations("data")
>>>> have enough disk spaces to store the uploaded file.
>>>>
>>>> Should we create a Jira for this? :-)
>>>>
>>>> Best Regards
>>>> --
>>>> Lin Gao
>>>> Software Engineer
>>>> JBoss by Red Hat
>>>>
>>>> ----- Original Message -----
>>>>> From: "Darran Lofthouse" <darran.lofthouse at jboss.com>
>>>>> To: "Jason T. Greene" <jason.greene at redhat.com>, "Tristan Tarrant" <ttarrant at redhat.com>
>>>>> Cc: wildfly-dev at lists.jboss.org
>>>>> Sent: Wednesday, November 4, 2015 11:21:11 PM
>>>>> Subject: Re: [wildfly-dev] Shall we limit size of the deployment in WildFly?
>>>>>
>>>>> This would also be something ideal to report on in the "administrator
>>>>> encouragement" service I keep meaning to look into again ;-)
>>>>>
>>>>> On 04/11/15 15:02, Jason T. Greene wrote:
>>>>>> Yes that would be very clear cut and easy to do.
>>>>>>
>>>>>>> On Nov 4, 2015, at 8:38 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>>>>>>
>>>>>>> While we're on filesystem space availability, for monitoring purposes,
>>>>>>> would it be possible to expose this as a runtime metric for known server
>>>>>>> locations (data, tmp, logs) ?
>>>>>>>
>>>>>>> Tristan
>>>>>>>
>>>>>>>> On 04/11/2015 13:37, Jason T. Greene wrote:
>>>>>>>> On Nov 3, 2015, at 7:14 PM, Lin Gao <lgao at redhat.com> wrote:
>>>>>>>>
>>>>>>>>>>> On 11/03/2015 02:41 AM, Lin Gao wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> WildFly does not limit the deployment file size, users with
>>>>>>>>>>> appropriate
>>>>>>>>>>> privilege(deployer) can select any file to deploy from both CLI and
>>>>>>>>>>> web
>>>>>>>>>>> console.
>>>>>>>>>>>
>>>>>>>>>>> For the too big file, it may exhaust all available disk space, and
>>>>>>>>>>> in
>>>>>>>>>>> some cases, even small file can exhaust the disk space if the
>>>>>>>>>>> current
>>>>>>>>>>> disk space is not big enough.
>>>>>>>>>>>
>>>>>>>>>>> So shall we limit file size of the deployment in WildFly? or shall
>>>>>>>>>>> we
>>>>>>>>>>> limit the available disk space? or shall we just show a warning
>>>>>>>>>>> message
>>>>>>>>>>> to users?
>>>>>>>>>>>
>>>>>>>>>>> If we do, where in the management API configuration for this should
>>>>>>>>>>> be
>>>>>>>>>>> done, if it is done this way?
>>>>>>>>>>>
>>>>>>>>>>> Arbitrary limits will break users, so if we have an arbitrary limit
>>>>>>>>>>> it
>>>>>>>>>>> needs to be easily adjusted.
>>>>>>>>>>>
>>>>>>>>>>> In case of domain mode, different hosts may have different disk
>>>>>>>>>>> space,
>>>>>>>>>>> which means they are likely have different capacity to hold
>>>>>>>>>>> deployment
>>>>>>>>>>> files. For example, servers in server-group-A may have 2GB
>>>>>>>>>>> available
>>>>>>>>>>> disk space, servers in server-group-B may have 200MB available disk
>>>>>>>>>>> space. An unified limit for the whole domain seems not fair for the
>>>>>>>>>>> servers with more available disk space.
>>>>>>>>>>>
>>>>>>>>>>> Also, WildFly does not limit type of the deployment file, but it
>>>>>>>>>>> might
>>>>>>>>>>> need a separate discussion if necessary?
>>>>>>>>>>>
>>>>>>>>>>> Any thoughts?
>>>>>>>>>>
>>>>>>>>>> Is this in response to a real observed problem?
>>>>>>>>>
>>>>>>>>> Yes it is.
>>>>>>>>
>>>>>>>> My understanding was that this was reported by a user but it was reported
>>>>>>>> as something they were concerned with, and not actually something that
>>>>>>>> was occurring in their environment.
>>>>>>>>
>>>>>>>> I agree with David that disk quotas on the processing running the app
>>>>>>>> server is the most appropriate solution to prevent interference with
>>>>>>>> other processes.
>>>>>>>>
>>>>>>>> I do not think we should have a hard coded limit as a constant because
>>>>>>>> that will prevent legit deployments from working, and you can still run
>>>>>>>> out of disk space (e.g you have 20k left and you upload a 21k
>>>>>>>> deployment, which is under 1G)
>>>>>>>>
>>>>>>>> We could have an extra check that verifies enough disk space in the add
>>>>>>>> content management op, however Java does not give us access to quota
>>>>>>>> information so the best we could do is only verify available disk space
>>>>>>>> (unless we exec a command for every platform we support).
>>>>>>>>
>>>>>>>> IMO this one is pretty low priority, but perhaps those that monitor this
>>>>>>>> list see it as a need. If so can you speak up?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> In general, if the user
>>>>>>>>>> doesn't have space for a deployment, the deployment will fail with an
>>>>>>>>>> error and (I am fairly certain) will delete the partial deployment. If
>>>>>>>>>> there is space, then the deployment will succeed regardless of size.
>>>>>>>>>
>>>>>>>>>> It's interesting that the JIRA you reference speaks in terms of
>>>>>>>>>> security. If an admin wants to lock down storage space, it's probably
>>>>>>>>>> best to do it at the operating system level using e.g. disk quotas -
>>>>>>>>>> there are too many ways to get the application server to write
>>>>>>>>>> arbitrary
>>>>>>>>>> amounts of data to the file system, regardless of the presence of a
>>>>>>>>>> security manager or any other application-level control.
>>>>>>>>>>
>>>>>>>>>> I'm pretty sure that if an attacker has permission to upload
>>>>>>>>>> deployments
>>>>>>>>>> to the server, they already essentially have control over the server.
>>>>>>>>>> This should be an OS level concern.
>>>>>>>>>
>>>>>>>>> The JIRA in question was a 'bug' related to security at first, after
>>>>>>>>> several
>>>>>>>>> rounds of discussion, it is agreed that it is not a security
>>>>>>>>> vulnerability, but
>>>>>>>>> an 'Enhancement'.
>>>>>>>>>
>>>>>>>>> The proposed requirement for the enhancement is:
>>>>>>>>>
>>>>>>>>> Provide an option in config to alert user that
>>>>>>>>> a) File is larger than a configurable limit
>>>>>>>>> b) File type is/is not valid.
>>>>>>>>>
>>>>>>>>> but it may need more discussion in community on both the requirement and
>>>>>>>>> design
>>>>>>>>> if it will be done, that is why this thread comes out in first place.
>>>>>>>>> :-)
>>>>>>>>>
>>>>>>>>>>> FYI: https://issues.jboss.org/browse/WFCORE-1057
>>>>>>>>>
>>>>>>>>> Best Regards
>>>>>>>>> --
>>>>>>>>> Lin Gao
>>>>>>>>> Software Engineer
>>>>>>>>> JBoss by Red Hat
>>>>>>>>> _______________________________________________
>>>>>>>>> wildfly-dev mailing list
>>>>>>>>> wildfly-dev at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> wildfly-dev mailing list
>>>>>>>> wildfly-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>>
>>>>>>> --
>>>>>>> Tristan Tarrant
>>>>>>> Infinispan Lead
>>>>>>> JBoss, a division of Red Hat
>>>>>>> _______________________________________________
>>>>>>> wildfly-dev mailing list
>>>>>>> wildfly-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>
>>>>>> _______________________________________________
>>>>>> wildfly-dev mailing list
>>>>>> wildfly-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>
>>>>> _______________________________________________
>>>>> wildfly-dev mailing list
>>>>> wildfly-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>
>>>
>>>
>>
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
More information about the wildfly-dev
mailing list