September 2014 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-3106) BufferManager Cleaner consuming 90% of total CPU time

by Devesh Mishra (JIRA)

[ https://issues.jboss.org/browse/TEIID-3106?page=com.atlassian.jira.plugin... ] Devesh Mishra commented on TEIID-3106: -------------------------------------- We haven't seen high cpu for last two days. Looks like the solutions works perfect. Steven, do you have further commits on this ? > BufferManager Cleaner consuming 90% of total CPU time > ----------------------------------------------------- > > Key: TEIID-3106 > URL: https://issues.jboss.org/browse/TEIID-3106 > Project: Teiid > Issue Type: Bug > Components: Query Engine > Affects Versions: 8.7 > Environment: zOS > Reporter: Devesh Mishra > Assignee: Steven Hawkins > Labels: teiid-engine > Attachments: BufferManagerImpl.java, BufferManagerImpl.java, CacheKey_Loog.patch, complete_log.zip, cpu monitoring.docx, dump.zip, in_local.docx, LrfuEvictionQueue.java, LrfuEvictionQueue.patch, non_high_cpu.docx > > > BufferManager Cleaner thread is consuming almost all of the CPU utilized by the jboss process. Thread dump shows following information. > 3XMTHREADINFO "BufferManager Cleaner" J9VMThread:0x0000004C41CFEB00, j9thread_t:0x0000004C52B85AE0, java/lang/Thread:0x000000481C036E20, state:CW, prio=5 > 3XMJAVALTHREAD (java/lang/Thread getId:0x76, isDaemon:true) > 3XMTHREADINFO1 (native thread ID:0x3AEC2600, native priority:0x5, native policy:UNKNOWN) > 3XMHEAPALLOC Heap bytes allocated since last GC cycle=2609184 (0x27D020) > 3XMTHREADINFO3 Java callstack: > 4XESTACKTRACE at java/util/concurrent/ConcurrentSkipListMap.doRemove(ConcurrentSkipListMap.java:1070(Compiled Code)) > 4XESTACKTRACE at java/util/concurrent/ConcurrentSkipListMap.remove(ConcurrentSkipListMap.java:1659(Compiled Code)) > 4XESTACKTRACE at org/teiid/common/buffer/impl/LrfuEvictionQueue.remove(LrfuEvictionQueue.java:60(Compiled Code)) > 4XESTACKTRACE at org/teiid/common/buffer/impl/BufferManagerImpl.doEvictions(BufferManagerImpl.java:854(Compiled Code)) > 5XESTACKTRACE (entered lock: org/teiid/common/buffer/CacheEntry@0x00000048393C2598, entry count: 1) > 4XESTACKTRACE at org/teiid/common/buffer/impl/BufferManagerImpl$Cleaner.run(BufferManagerImpl.java:108) > 4XESTACKTRACE at java/util/TimerThread.mainLoop(Timer.java:555) > 4XESTACKTRACE at java/util/TimerThread.run(Timer.java:505) > When we added log statements around the BufferManagerImpl.doEvictions() it loops through the remove and firstEntry loop. > Forum discussion link : https://community.jboss.org/message/901792 -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2848) MongoDB: Provide Array support

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2848?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2848: --------------------------------------- See https://docs.jboss.org/author/display/TEIID/ARRAYTABLE > MongoDB: Provide Array support > ------------------------------- > > Key: TEIID-2848 > URL: https://issues.jboss.org/browse/TEIID-2848 > Project: Teiid > Issue Type: Enhancement > Components: Misc. Connectors > Reporter: Ramesh Reddy > Assignee: Ramesh Reddy > Labels: CR1 > Fix For: 8.8, 8.7.1 > > > Provide array support for primitive data types in MongoDB translator. The arrays are supported but as embedded documents in the one-2-many situation. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3070) Netty worker threads cause system high cpu

by Kylin Soong (JIRA)

[ https://issues.jboss.org/browse/TEIID-3070?page=com.atlassian.jira.plugin... ] Kylin Soong commented on TEIID-3070: ------------------------------------ Norman have a fix https://github.com/netty/netty/pull/2868, we need upgrade netty after norman add the fix to a final release. > Netty worker threads cause system high cpu > ------------------------------------------ > > Key: TEIID-3070 > URL: https://issues.jboss.org/browse/TEIID-3070 > Project: Teiid > Issue Type: Bug > Components: Server > Affects Versions: 8.7 > Environment: JDV 6.1.0.DR2 > JDV 6.0 > Reporter: Kylin Soong > Assignee: Kylin Soong > > I have hit this issue many times in deploying a VDB to JDV, All CPU be used by netty worker threads: > ~~~ > "New I/O worker #3" daemon prio=10 tid=0x00007fc71c0d2000 nid=0x320b runnable [0x00007fc7074f3000] > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) > - locked <0x00000000c42b4318> (a sun.nio.ch.Util$2) > - locked <0x00000000c42b4328> (a java.util.Collections$UnmodifiableSet) > - locked <0x00000000c42b42d0> (a sun.nio.ch.EPollSelectorImpl) > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) > at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:64) > ~~~ > Seems this is a exist netty issue, search "jboss netty high cpu" via google we can get lots of result. > Below link have detailed depiction about this issue; > https://github.com/kylinsoong/teiid-samples/tree/master/highcpu -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2831) Add a full ddl example to the ldap translator docs

by Kylin Soong (JIRA)

[ https://issues.jboss.org/browse/TEIID-2831?page=com.atlassian.jira.plugin... ] Kylin Soong reassigned TEIID-2831: ---------------------------------- Assignee: Kylin Soong > Add a full ddl example to the ldap translator docs > -------------------------------------------------- > > Key: TEIID-2831 > URL: https://issues.jboss.org/browse/TEIID-2831 > Project: Teiid > Issue Type: Task > Components: Documentation > Reporter: Steven Hawkins > Assignee: Kylin Soong > > It would be helpful to have a full example of LDAP metadata. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2848) MongoDB: Provide Array support

by Ivan Chan (JIRA)

[ https://issues.jboss.org/browse/TEIID-2848?page=com.atlassian.jira.plugin... ] Ivan Chan edited comment on TEIID-2848 at 9/8/14 7:02 PM: ---------------------------------------------------------- Where can I find document of using ARRAYTABLE and Object[]? Thanks! I want to try it out. was (Author: ichanjasper): Where can I find document of using ARRAYTABLE and Object[]? Thanks! > MongoDB: Provide Array support > ------------------------------- > > Key: TEIID-2848 > URL: https://issues.jboss.org/browse/TEIID-2848 > Project: Teiid > Issue Type: Enhancement > Components: Misc. Connectors > Reporter: Ramesh Reddy > Assignee: Ramesh Reddy > Labels: CR1 > Fix For: 8.8, 8.7.1 > > > Provide array support for primitive data types in MongoDB translator. The arrays are supported but as embedded documents in the one-2-many situation. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2848) MongoDB: Provide Array support

by Ivan Chan (JIRA)

[ https://issues.jboss.org/browse/TEIID-2848?page=com.atlassian.jira.plugin... ] Ivan Chan commented on TEIID-2848: ---------------------------------- Where can I find document of using ARRAYTABLE and Object[]? Thanks! > MongoDB: Provide Array support > ------------------------------- > > Key: TEIID-2848 > URL: https://issues.jboss.org/browse/TEIID-2848 > Project: Teiid > Issue Type: Enhancement > Components: Misc. Connectors > Reporter: Ramesh Reddy > Assignee: Ramesh Reddy > Labels: CR1 > Fix For: 8.8, 8.7.1 > > > Provide array support for primitive data types in MongoDB translator. The arrays are supported but as embedded documents in the one-2-many situation. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3119) Improve full sort performance for dup removal and grouping

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3119?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-3119: --------------------------------------- Updated dup removal to use tree logic instead of the progressive multi-pass sort. This should be better in almost all scenarios as any recopy of the result is offset by not having to buffer the input and not perform additional passes. > Improve full sort performance for dup removal and grouping > ---------------------------------------------------------- > > Key: TEIID-3119 > URL: https://issues.jboss.org/browse/TEIID-3119 > Project: Teiid > Issue Type: Enhancement > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Fix For: 8.9 > > > We perform a full sort for grouping/distinct reusing the existing sorting logic. This is not as efficient as it could for larger data sets with smaller numbers of groups. A hash implementation or combining the grouping with the sorting would perform better. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3106) BufferManager Cleaner consuming 90% of total CPU time

by Devesh Mishra (JIRA)

[ https://issues.jboss.org/browse/TEIID-3106?page=com.atlassian.jira.plugin... ] Devesh Mishra commented on TEIID-3106: -------------------------------------- Thanks Steven, I have applied the commit and testing it. so far so good. no high cpu till now. Lets hope for the best. > BufferManager Cleaner consuming 90% of total CPU time > ----------------------------------------------------- > > Key: TEIID-3106 > URL: https://issues.jboss.org/browse/TEIID-3106 > Project: Teiid > Issue Type: Bug > Components: Query Engine > Affects Versions: 8.7 > Environment: zOS > Reporter: Devesh Mishra > Assignee: Steven Hawkins > Labels: teiid-engine > Attachments: BufferManagerImpl.java, BufferManagerImpl.java, CacheKey_Loog.patch, complete_log.zip, cpu monitoring.docx, dump.zip, in_local.docx, LrfuEvictionQueue.java, LrfuEvictionQueue.patch, non_high_cpu.docx > > > BufferManager Cleaner thread is consuming almost all of the CPU utilized by the jboss process. Thread dump shows following information. > 3XMTHREADINFO "BufferManager Cleaner" J9VMThread:0x0000004C41CFEB00, j9thread_t:0x0000004C52B85AE0, java/lang/Thread:0x000000481C036E20, state:CW, prio=5 > 3XMJAVALTHREAD (java/lang/Thread getId:0x76, isDaemon:true) > 3XMTHREADINFO1 (native thread ID:0x3AEC2600, native priority:0x5, native policy:UNKNOWN) > 3XMHEAPALLOC Heap bytes allocated since last GC cycle=2609184 (0x27D020) > 3XMTHREADINFO3 Java callstack: > 4XESTACKTRACE at java/util/concurrent/ConcurrentSkipListMap.doRemove(ConcurrentSkipListMap.java:1070(Compiled Code)) > 4XESTACKTRACE at java/util/concurrent/ConcurrentSkipListMap.remove(ConcurrentSkipListMap.java:1659(Compiled Code)) > 4XESTACKTRACE at org/teiid/common/buffer/impl/LrfuEvictionQueue.remove(LrfuEvictionQueue.java:60(Compiled Code)) > 4XESTACKTRACE at org/teiid/common/buffer/impl/BufferManagerImpl.doEvictions(BufferManagerImpl.java:854(Compiled Code)) > 5XESTACKTRACE (entered lock: org/teiid/common/buffer/CacheEntry@0x00000048393C2598, entry count: 1) > 4XESTACKTRACE at org/teiid/common/buffer/impl/BufferManagerImpl$Cleaner.run(BufferManagerImpl.java:108) > 4XESTACKTRACE at java/util/TimerThread.mainLoop(Timer.java:555) > 4XESTACKTRACE at java/util/TimerThread.run(Timer.java:505) > When we added log statements around the BufferManagerImpl.doEvictions() it loops through the remove and firstEntry loop. > Forum discussion link : https://community.jboss.org/message/901792 -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3119) Improve full sort performance for dup removal and grouping

by Steven Hawkins (JIRA)

Steven Hawkins created TEIID-3119: ------------------------------------- Summary: Improve full sort performance for dup removal and grouping Key: TEIID-3119 URL: https://issues.jboss.org/browse/TEIID-3119 Project: Teiid Issue Type: Enhancement Components: Query Engine Reporter: Steven Hawkins Assignee: Steven Hawkins Fix For: 8.9 We perform a full sort for grouping/distinct reusing the existing sorting logic. This is not as efficient as it could for larger data sets with smaller numbers of groups. A hash implementation or combining the grouping with the sorting would perform better. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3106) BufferManager Cleaner consuming 90% of total CPU time

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3106?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-3106: --------------------------------------- I should add that there were some additional changes in the last commit related to simplifications and parameter tuning that have seemed beneficial as this has been locally tested. I can also attach a simplified patch if needed as well. > BufferManager Cleaner consuming 90% of total CPU time > ----------------------------------------------------- > > Key: TEIID-3106 > URL: https://issues.jboss.org/browse/TEIID-3106 > Project: Teiid > Issue Type: Bug > Components: Query Engine > Affects Versions: 8.7 > Environment: zOS > Reporter: Devesh Mishra > Assignee: Steven Hawkins > Labels: teiid-engine > Attachments: BufferManagerImpl.java, BufferManagerImpl.java, CacheKey_Loog.patch, complete_log.zip, cpu monitoring.docx, dump.zip, in_local.docx, LrfuEvictionQueue.java, LrfuEvictionQueue.patch, non_high_cpu.docx > > > BufferManager Cleaner thread is consuming almost all of the CPU utilized by the jboss process. Thread dump shows following information. > 3XMTHREADINFO "BufferManager Cleaner" J9VMThread:0x0000004C41CFEB00, j9thread_t:0x0000004C52B85AE0, java/lang/Thread:0x000000481C036E20, state:CW, prio=5 > 3XMJAVALTHREAD (java/lang/Thread getId:0x76, isDaemon:true) > 3XMTHREADINFO1 (native thread ID:0x3AEC2600, native priority:0x5, native policy:UNKNOWN) > 3XMHEAPALLOC Heap bytes allocated since last GC cycle=2609184 (0x27D020) > 3XMTHREADINFO3 Java callstack: > 4XESTACKTRACE at java/util/concurrent/ConcurrentSkipListMap.doRemove(ConcurrentSkipListMap.java:1070(Compiled Code)) > 4XESTACKTRACE at java/util/concurrent/ConcurrentSkipListMap.remove(ConcurrentSkipListMap.java:1659(Compiled Code)) > 4XESTACKTRACE at org/teiid/common/buffer/impl/LrfuEvictionQueue.remove(LrfuEvictionQueue.java:60(Compiled Code)) > 4XESTACKTRACE at org/teiid/common/buffer/impl/BufferManagerImpl.doEvictions(BufferManagerImpl.java:854(Compiled Code)) > 5XESTACKTRACE (entered lock: org/teiid/common/buffer/CacheEntry@0x00000048393C2598, entry count: 1) > 4XESTACKTRACE at org/teiid/common/buffer/impl/BufferManagerImpl$Cleaner.run(BufferManagerImpl.java:108) > 4XESTACKTRACE at java/util/TimerThread.mainLoop(Timer.java:555) > 4XESTACKTRACE at java/util/TimerThread.run(Timer.java:505) > When we added log statements around the BufferManagerImpl.doEvictions() it loops through the remove and firstEntry loop. > Forum discussion link : https://community.jboss.org/message/901792 -- This message was sent by Atlassian JIRA (v6.3.1#6329)

11 years, 7 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues September 2014