July 2010 - teiid-commits - Jboss List Archives

Teiid-NightlyBuild - Build # 900 - Unstable!

by jboss-qa-internal＠redhat.com

Teiid-NightlyBuild - Build # 900 - Unstable: Check console output at http://hudson.qa.jboss.com/hudson/job/Teiid-NightlyBuild/900/ to view the results.

14 years, 4 months

1
9
0 / 0

teiid SVN: r2395 - in trunk: connectors/translator-ws/src/main/java/org/teiid/translator/ws and 22 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2010-07-31 18:31:31 -0400 (Sat, 31 Jul 2010) New Revision: 2395 Added: trunk/engine/src/main/java/org/teiid/query/optimizer/ProcedurePlanner.java trunk/engine/src/main/java/org/teiid/query/processor/BatchedUpdatePlan.java trunk/engine/src/main/java/org/teiid/query/processor/proc/Program.java trunk/engine/src/main/java/org/teiid/query/processor/proc/ProgramInstruction.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestBatchedUpdatePlanner.java trunk/engine/src/test/java/org/teiid/query/processor/TestBatchedUpdatePlan.java Removed: trunk/engine/src/main/java/org/teiid/query/optimizer/proc/ trunk/engine/src/main/java/org/teiid/query/processor/batch/ trunk/engine/src/main/java/org/teiid/query/processor/program/ trunk/engine/src/main/java/org/teiid/query/util/XMLFormatConstants.java trunk/engine/src/test/java/org/teiid/query/optimizer/batch/ trunk/engine/src/test/java/org/teiid/query/processor/batch/ Modified: trunk/client/src/main/java/org/teiid/jdbc/DatabaseMetaDataImpl.java trunk/connectors/translator-ws/src/main/java/org/teiid/translator/ws/WSProcedureExecution.java trunk/engine/src/main/java/org/teiid/common/buffer/BufferManager.java trunk/engine/src/main/java/org/teiid/common/buffer/SPage.java trunk/engine/src/main/java/org/teiid/common/buffer/STree.java trunk/engine/src/main/java/org/teiid/common/buffer/impl/BufferManagerImpl.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/DQPCore.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadata.java trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadataWrapper.java trunk/engine/src/main/java/org/teiid/query/metadata/QueryMetadataInterface.java trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java trunk/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java trunk/engine/src/main/java/org/teiid/query/processor/ProcessorDataManager.java trunk/engine/src/main/java/org/teiid/query/processor/TempTableDataManager.java trunk/engine/src/main/java/org/teiid/query/processor/proc/AssignmentInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/BreakInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/CreateCursorResultSetInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/ErrorInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/ExecDynamicSqlInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/IfInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/LoopInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java trunk/engine/src/main/java/org/teiid/query/processor/proc/RepeatedInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/proc/WhileInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/relational/AccessNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/BatchedUpdateNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentProcedureAccessNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/PlanExecutionNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/SubqueryAwareEvaluator.java trunk/engine/src/main/java/org/teiid/query/processor/xml/InitializeDocumentInstruction.java trunk/engine/src/main/java/org/teiid/query/processor/xml/RelationalPlanExecutor.java trunk/engine/src/main/java/org/teiid/query/processor/xml/XMLPlan.java trunk/engine/src/main/java/org/teiid/query/sql/lang/From.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/GroupSymbol.java trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java trunk/engine/src/test/java/org/teiid/common/buffer/TestSTree.java trunk/engine/src/test/java/org/teiid/dqp/internal/process/SimpleQueryProcessorFactory.java trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java trunk/engine/src/test/java/org/teiid/query/processor/FakeDataManager.java trunk/engine/src/test/java/org/teiid/query/processor/HardcodedDataManager.java trunk/engine/src/test/java/org/teiid/query/processor/TestProcedureRelational.java trunk/engine/src/test/java/org/teiid/query/processor/TestTempTables.java trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java trunk/engine/src/test/java/org/teiid/query/unittest/FakeMetadataFactory.java Log: TEIID-168 beginning to rough in support for temp tables that are global to the vdb. also minor cleanups. Modified: trunk/client/src/main/java/org/teiid/jdbc/DatabaseMetaDataImpl.java =================================================================== --- trunk/client/src/main/java/org/teiid/jdbc/DatabaseMetaDataImpl.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/client/src/main/java/org/teiid/jdbc/DatabaseMetaDataImpl.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -263,8 +263,8 @@ .append(".ProcedureParams as p") //$NON-NLS-1$ .append(" WHERE UCASE(VDBName)").append(LIKE_ESCAPE)//$NON-NLS-1$ .append(" AND UCASE(SchemaName)").append(LIKE_ESCAPE)//$NON-NLS-1$ - .append(" AND UCASE(ProcedureName)") .append(LIKE_ESCAPE) //$NON-NLS-1$ - .append(" AND UCASE(p.Name) LIKE ?") //$NON-NLS-1$ + .append(" AND UCASE(ProcedureName)").append(LIKE_ESCAPE) //$NON-NLS-1$ + .append(" AND UCASE(p.Name)").append(LIKE_ESCAPE) //$NON-NLS-1$ .append(" ORDER BY PROCEDURE_SCHEM, PROCEDURE_NAME, COLUMN_TYPE, POSITION").toString(); //$NON-NLS-1$ private static final String QUERY_SCHEMAS = Modified: trunk/connectors/translator-ws/src/main/java/org/teiid/translator/ws/WSProcedureExecution.java =================================================================== --- trunk/connectors/translator-ws/src/main/java/org/teiid/translator/ws/WSProcedureExecution.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/connectors/translator-ws/src/main/java/org/teiid/translator/ws/WSProcedureExecution.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -85,7 +85,7 @@ XMLType docObject = (XMLType)arguments.get(2).getArgumentValue().getValue(); Source source = null; try { - source = converToSource(docObject); + source = convertToSource(docObject); String endpoint = (String)arguments.get(3).getArgumentValue().getValue(); if (style == null) { @@ -149,7 +149,7 @@ } } - private StreamSource converToSource(SQLXML xml) throws SQLException { + private StreamSource convertToSource(SQLXML xml) throws SQLException { if (xml == null) { return null; } Modified: trunk/engine/src/main/java/org/teiid/common/buffer/BufferManager.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/BufferManager.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/common/buffer/BufferManager.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -114,6 +114,6 @@ */ int getSchemaSize(List elements); - STree createSTree(final List elements, String groupName, TupleSourceType tupleSourceType, int keyLength); + STree createSTree(final List elements, String groupName, int keyLength); } Modified: trunk/engine/src/main/java/org/teiid/common/buffer/SPage.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/SPage.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/common/buffer/SPage.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -84,23 +84,31 @@ return new SearchResult(-previousValues.getTuples().size() - 1, page.prev, previousValues); } if (parent != null && index != 0) { - //for non-matches move the previous pointer over to this page - SPage childPage = page; - List oldKey = null; - List newKey = page.stree.extractKey(values.getTuples().get(0)); - for (Iterator<SearchResult> desc = parent.descendingIterator(); desc.hasNext();) { - SearchResult sr = desc.next(); - int parentIndex = Math.max(0, -sr.index - 2); - if (oldKey == null) { - oldKey = sr.values.getTuples().set(parentIndex, newKey); - } else if (page.stree.comparator.compare(oldKey, sr.values.getTuples().get(parentIndex)) == 0 ) { - sr.values.getTuples().set(parentIndex, newKey); - } else { - break; + page.stree.updateLock.lock(); + try { + index = Collections.binarySearch(values.getTuples(), k, page.stree.comparator); + if (index != 0) { + //for non-matches move the previous pointer over to this page + SPage childPage = page; + List oldKey = null; + List newKey = page.stree.extractKey(values.getTuples().get(0)); + for (Iterator<SearchResult> desc = parent.descendingIterator(); desc.hasNext();) { + SearchResult sr = desc.next(); + int parentIndex = Math.max(0, -sr.index - 2); + if (oldKey == null) { + oldKey = sr.values.getTuples().set(parentIndex, newKey); + } else if (page.stree.comparator.compare(oldKey, sr.values.getTuples().get(parentIndex)) == 0 ) { + sr.values.getTuples().set(parentIndex, newKey); + } else { + break; + } + sr.page.children.set(parentIndex, childPage); + sr.page.setValues(sr.values); + childPage = sr.page; + } } - sr.page.children.set(parentIndex, childPage); - sr.page.setValues(sr.values); - childPage = sr.page; + } finally { + page.stree.updateLock.unlock(); } } } Modified: trunk/engine/src/main/java/org/teiid/common/buffer/STree.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -29,6 +29,7 @@ import java.util.List; import java.util.Random; import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.locks.ReentrantLock; import org.teiid.common.buffer.SPage.SearchResult; import org.teiid.core.TeiidComponentException; @@ -47,7 +48,7 @@ private int mask = 1; private int shift = 1; - protected SPage[] header = new SPage[] {new SPage(this, true)}; + protected volatile SPage[] header = new SPage[] {new SPage(this, true)}; protected BatchManager keyManager; protected BatchManager leafManager; protected Comparator comparator; @@ -56,6 +57,8 @@ protected String[] types; protected String[] keytypes; + protected ReentrantLock updateLock = new ReentrantLock(); + private AtomicInteger rowCount = new AtomicInteger(); public STree(BatchManager manager, Modified: trunk/engine/src/main/java/org/teiid/common/buffer/impl/BufferManagerImpl.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/impl/BufferManagerImpl.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/common/buffer/impl/BufferManagerImpl.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -472,7 +472,7 @@ return tupleBuffer; } - public STree createSTree(final List elements, String groupName, TupleSourceType tupleSourceType, int keyLength) { + public STree createSTree(final List elements, String groupName, int keyLength) { String newID = String.valueOf(this.tsId.getAndIncrement()); int[] lobIndexes = LobManager.getLobIndexes(elements); BatchManager bm = new BatchManagerImpl(newID, elements.size(), lobIndexes); @@ -481,7 +481,7 @@ for (int i = 1; i < compareIndexes.length; i++) { compareIndexes[i] = i; } - LogManager.logDetail(LogConstants.CTX_BUFFER_MGR, "Creating STree:", newID, "of type ", tupleSourceType); //$NON-NLS-1$ //$NON-NLS-2$ + LogManager.logDetail(LogConstants.CTX_BUFFER_MGR, "Creating STree:", newID); //$NON-NLS-1$ return new STree(keyManager, bm, new ListNestedSortComparator(compareIndexes), getProcessorBatchSize(), keyLength, TupleBuffer.getTypeNames(elements)); } Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/DQPCore.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/DQPCore.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/DQPCore.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -25,7 +25,6 @@ import java.util.ArrayList; import java.util.Collection; import java.util.Collections; -import java.util.HashMap; import java.util.LinkedList; import java.util.List; import java.util.Map; @@ -146,10 +145,10 @@ static class ClientState { List<RequestID> requests; - TempTableStore tempTableStoreImpl; + TempTableStore sessionTables; public ClientState(TempTableStore tableStoreImpl) { - this.tempTableStoreImpl = tableStoreImpl; + this.sessionTables = tableStoreImpl; } public synchronized void addRequest(RequestID requestID) { @@ -203,7 +202,7 @@ private int chunkSize = Streamable.STREAMING_BATCH_SIZE_IN_BYTES; private Map<RequestID, RequestWorkItem> requests = new ConcurrentHashMap<RequestID, RequestWorkItem>(); - private Map<String, ClientState> clientState = Collections.synchronizedMap(new HashMap<String, ClientState>()); + private Map<String, ClientState> clientState = new ConcurrentHashMap<String, ClientState>(); private boolean useEntitlements = false; private int maxActivePlans = DQPConfiguration.DEFAULT_MAX_ACTIVE_PLANS; @@ -236,14 +235,12 @@ } public ClientState getClientState(String key, boolean create) { - synchronized (clientState) { - ClientState state = clientState.get(key); - if (state == null && create) { - state = new ClientState(new TempTableStore(bufferManager, key, null)); - clientState.put(key, state); - } - return state; + ClientState state = clientState.get(key); + if (state == null && create) { + state = new ClientState(new TempTableStore(bufferManager, key)); + clientState.put(key, state); } + return state; } /** @@ -337,7 +334,7 @@ ClientState state = this.getClientState(workContext.getSessionId(), true); request.initialize(requestMsg, bufferManager, dataTierMgr, transactionService, processorDebugAllowed, - state.tempTableStoreImpl, workContext, + state.sessionTables, workContext, connectorManagerRepository, this.useEntitlements); ResultsFuture<ResultsMessage> resultsFuture = new ResultsFuture<ResultsMessage>(); @@ -675,8 +672,7 @@ this.bufferService, this.maxCodeTables, this.maxCodeRecords, - this.maxCodeTableRecords); - + this.maxCodeTableRecords); } public void setBufferService(BufferService service) { Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -126,14 +126,14 @@ return this.connectorManagerRepository.getConnectorManager(connectorName); } - public TupleSource registerRequest(Object processorId, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException { - RequestWorkItem workItem = requestMgr.getRequestWorkItem((RequestID)processorId); + public TupleSource registerRequest(CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException { + RequestWorkItem workItem = requestMgr.getRequestWorkItem((RequestID)context.getProcessorID()); if(CoreConstants.SYSTEM_MODEL.equals(modelName)) { return processSystemQuery(command, workItem.getDqpWorkContext()); } - AtomicRequestMessage aqr = createRequest(processorId, command, modelName, connectorBindingId, nodeID); + AtomicRequestMessage aqr = createRequest(context.getProcessorID(), command, modelName, connectorBindingId, nodeID); ConnectorWork work = getCM(aqr.getConnectorName()).registerRequest(aqr); return new DataTierTupleSource(aqr, workItem, work, this); } Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -43,7 +43,6 @@ import org.teiid.dqp.internal.process.SessionAwareCache.CacheID; import org.teiid.dqp.internal.process.multisource.MultiSourceMetadataWrapper; import org.teiid.dqp.message.RequestID; -import org.teiid.language.SQLConstants.NonReserved; import org.teiid.query.analysis.AnalysisRecord; import org.teiid.query.metadata.QueryMetadataInterface; import org.teiid.query.metadata.SupportConstants; @@ -121,7 +120,7 @@ if (workItem != null) { ClientState state = requestManager.getClientState(workContext.getSessionId(), false); if (state != null) { - tempTableStore = state.tempTableStoreImpl; + tempTableStore = state.sessionTables; } } } @@ -389,7 +388,7 @@ } public Map<Integer, Object> getDefaultColumn(String tableName, String columnName, - Class javaType) { + Class<?> javaType) { Map<Integer, Object> column = new HashMap<Integer, Object>(); Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -28,6 +28,7 @@ import java.util.Properties; import java.util.Set; +import org.teiid.adminapi.impl.VDBMetaData; import org.teiid.api.exception.query.QueryMetadataException; import org.teiid.api.exception.query.QueryParserException; import org.teiid.api.exception.query.QueryPlannerException; @@ -130,6 +131,7 @@ protected Command userCommand; protected boolean returnsUpdateCount; protected boolean useEntitlements; + private TempTableStore globalTables; void initialize(RequestMessage requestMsg, BufferManager bufferManager, @@ -175,7 +177,18 @@ this.capabilitiesFinder = new CachedFinder(this.connectorManagerRepo, workContext.getVDB()); this.capabilitiesFinder = new TempCapabilitiesFinder(this.capabilitiesFinder); - metadata = workContext.getVDB().getAttachment(QueryMetadataInterface.class); + VDBMetaData vdbMetadata = workContext.getVDB(); + metadata = vdbMetadata.getAttachment(QueryMetadataInterface.class); + globalTables = vdbMetadata.getAttachment(TempTableStore.class); + if (globalTables == null) { + synchronized (vdbMetadata) { + globalTables = vdbMetadata.getAttachment(TempTableStore.class); + if (globalTables == null) { + globalTables = new TempTableStore(bufferManager, "SYSTEM"); //$NON-NLS-1$ + vdbMetadata.addAttchment(TempTableStore.class, globalTables); + } + } + } if (metadata == null) { throw new TeiidComponentException(DQPPlugin.Util.getString("DQPCore.Unable_to_load_metadata_for_VDB_name__{0},_version__{1}", this.vdbName, this.vdbVersion)); //$NON-NLS-1$ @@ -234,7 +247,7 @@ this.requestMsg.getShowPlan() != ShowPlan.OFF); this.context.setProcessorBatchSize(bufferManager.getProcessorBatchSize()); this.context.setConnectorBatchSize(bufferManager.getConnectorBatchSize()); - + this.context.setGlobalTableStore(this.globalTables); if (multiSourceModels != null) { MultiSourcePlanToProcessConverter modifier = new MultiSourcePlanToProcessConverter( metadata, idGenerator, analysisRecord, capabilitiesFinder, @@ -470,7 +483,7 @@ validateQuery(newCommand); - CommandContext copy = (CommandContext) commandContext.clone(); + CommandContext copy = commandContext.clone(); if (recursionGroup != null) { copy.pushCall(recursionGroup); } Modified: trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadata.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadata.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadata.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -503,5 +503,10 @@ public FunctionLibrary getFunctionLibrary() { return null; } + + @Override + public boolean isPrimaryKey(Object metadataID) { + return false; + } } Modified: trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadataWrapper.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadataWrapper.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/metadata/BasicQueryMetadataWrapper.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -368,5 +368,10 @@ public FunctionLibrary getFunctionLibrary() { return actualMetadata.getFunctionLibrary(); } + + @Override + public boolean isPrimaryKey(Object metadataID) { + return actualMetadata.isPrimaryKey(metadataID); + } } Modified: trunk/engine/src/main/java/org/teiid/query/metadata/QueryMetadataInterface.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/QueryMetadataInterface.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/metadata/QueryMetadataInterface.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -678,4 +678,6 @@ throws TeiidComponentException, QueryMetadataException; FunctionLibrary getFunctionLibrary(); + + boolean isPrimaryKey(Object metadataID); } Modified: trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -692,5 +692,17 @@ return this.actualMetadata.isScalarGroup(groupID); } + + @Override + public boolean isPrimaryKey(Object metadataID) { + if (metadataID instanceof TempMetadataID) { + TempMetadataID tid = (TempMetadataID)metadataID; + if (tid.getPrimaryKey() != null) { + return true; + } + return false; + } + return this.actualMetadata.isPrimaryKey(metadataID); + } } Modified: trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -49,7 +49,7 @@ private boolean isTempTable; private Collection<TempMetadataID> accessPatterns; private int cardinality = QueryMetadataInterface.UNKNOWN_CARDINALITY; - private LRUCache localCache; + private LRUCache<Object, Object> localCache; private boolean scalarGroup; private List<TempMetadataID> primaryKey; @@ -234,7 +234,7 @@ Object setProperty(Object key, Object value) { if (this.localCache == null) { - this.localCache = new LRUCache(LOCAL_CACHE_SIZE); + this.localCache = new LRUCache<Object, Object>(LOCAL_CACHE_SIZE); } return this.localCache.put(key, value); } Modified: trunk/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -1067,4 +1067,11 @@ public FunctionLibrary getFunctionLibrary() { return this.functionLibrary; } + + @Override + public boolean isPrimaryKey(Object metadataID) { + ArgCheck.isInstanceOf(KeyRecord.class, metadataID); + KeyRecord key = (KeyRecord)metadataID; + return key.getType() == org.teiid.metadata.KeyRecord.Type.Primary; + } } \ No newline at end of file Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -38,8 +38,8 @@ import org.teiid.query.optimizer.capabilities.SourceCapabilities; import org.teiid.query.optimizer.capabilities.SourceCapabilities.Capability; import org.teiid.query.optimizer.relational.rules.CapabilitiesUtil; +import org.teiid.query.processor.BatchedUpdatePlan; import org.teiid.query.processor.ProcessorPlan; -import org.teiid.query.processor.batch.BatchedUpdatePlan; import org.teiid.query.processor.relational.BatchedUpdateNode; import org.teiid.query.processor.relational.ProjectNode; import org.teiid.query.processor.relational.RelationalPlan; Copied: trunk/engine/src/main/java/org/teiid/query/optimizer/ProcedurePlanner.java (from rev 2390, trunk/engine/src/main/java/org/teiid/query/optimizer/proc/ProcedurePlanner.java) =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/ProcedurePlanner.java (rev 0) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/ProcedurePlanner.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -0,0 +1,325 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.optimizer; + +import java.util.Map; + +import org.teiid.api.exception.query.QueryMetadataException; +import org.teiid.api.exception.query.QueryPlannerException; +import org.teiid.core.TeiidComponentException; +import org.teiid.core.id.IDGenerator; +import org.teiid.core.util.Assertion; +import org.teiid.query.analysis.AnalysisRecord; +import org.teiid.query.execution.QueryExecPlugin; +import org.teiid.query.metadata.QueryMetadataInterface; +import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; +import org.teiid.query.processor.ProcessorPlan; +import org.teiid.query.processor.proc.AssignmentInstruction; +import org.teiid.query.processor.proc.BreakInstruction; +import org.teiid.query.processor.proc.ContinueInstruction; +import org.teiid.query.processor.proc.CreateCursorResultSetInstruction; +import org.teiid.query.processor.proc.ErrorInstruction; +import org.teiid.query.processor.proc.ExecDynamicSqlInstruction; +import org.teiid.query.processor.proc.IfInstruction; +import org.teiid.query.processor.proc.LoopInstruction; +import org.teiid.query.processor.proc.ProcedurePlan; +import org.teiid.query.processor.proc.Program; +import org.teiid.query.processor.proc.ProgramInstruction; +import org.teiid.query.processor.proc.WhileInstruction; +import org.teiid.query.sql.lang.Command; +import org.teiid.query.sql.lang.DynamicCommand; +import org.teiid.query.sql.lang.ProcedureContainer; +import org.teiid.query.sql.lang.StoredProcedure; +import org.teiid.query.sql.lang.TranslatableProcedureContainer; +import org.teiid.query.sql.proc.AssignmentStatement; +import org.teiid.query.sql.proc.Block; +import org.teiid.query.sql.proc.CommandStatement; +import org.teiid.query.sql.proc.CreateUpdateProcedureCommand; +import org.teiid.query.sql.proc.IfStatement; +import org.teiid.query.sql.proc.LoopStatement; +import org.teiid.query.sql.proc.RaiseErrorStatement; +import org.teiid.query.sql.proc.Statement; +import org.teiid.query.sql.proc.WhileStatement; +import org.teiid.query.sql.symbol.Expression; +import org.teiid.query.sql.symbol.ScalarSubquery; +import org.teiid.query.sql.visitor.CommandCollectorVisitor; +import org.teiid.query.util.CommandContext; + + +/** + * <p> This prepares an {@link org.teiid.query.processor.proc.ProcedurePlan ProcedurePlan} from + * a CreateUpdateProcedureCommand {@link org.teiid.query.sql.proc.CreateUpdateProcedureCommand CreateUpdateProcedureCommand}. + * </p> + */ +public final class ProcedurePlanner implements CommandPlanner { + + /** + * <p>Produce a ProcessorPlan for the CreateUpdateProcedureCommand on the current node + * of the CommandTreeNode, the procedure plan construction involves using the child + * processor plans.</p> + * @param metadata source of metadata + * @param debug whether or not to generate verbose debug output during planning + * @return ProcessorPlan This processorPlan is a <code>ProcedurePlan</code> + * @throws QueryPlannerException indicating a problem in planning + * @throws QueryMetadataException indicating an exception in accessing the metadata + * @throws TeiidComponentException indicating an unexpected exception + */ + public ProcessorPlan optimize(Command procCommand, IDGenerator idGenerator, QueryMetadataInterface metadata, CapabilitiesFinder capFinder, AnalysisRecord analysisRecord, CommandContext context) + throws QueryPlannerException, QueryMetadataException, TeiidComponentException { + + boolean debug = analysisRecord.recordDebug(); + if(debug) { + analysisRecord.println("\n####################################################"); //$NON-NLS-1$ + analysisRecord.println("PROCEDURE COMMAND: " + procCommand); //$NON-NLS-1$ + } + + Assertion.isInstanceOf(procCommand, CreateUpdateProcedureCommand.class, "Wrong command type"); //$NON-NLS-1$ + + if(debug) { + analysisRecord.println("OPTIMIZING SUB-COMMANDS: "); //$NON-NLS-1$ + } + + for (Command command : CommandCollectorVisitor.getCommands(procCommand)) { + if (!(command instanceof DynamicCommand)) { + command.setProcessorPlan(QueryOptimizer.optimizePlan(command, metadata, idGenerator, capFinder, analysisRecord, context)); + } + } + + Block block = ((CreateUpdateProcedureCommand) procCommand).getBlock(); + + Program programBlock = planBlock(((CreateUpdateProcedureCommand)procCommand), block, metadata, debug, idGenerator, capFinder, analysisRecord, context); + + if(debug) { + analysisRecord.println("\n####################################################"); //$NON-NLS-1$ + } + + // create plan from program and initialized environment + ProcedurePlan plan = new ProcedurePlan(programBlock); + + // propagate procedure parameters to the plan to allow runtime type checking + ProcedureContainer container = (ProcedureContainer)((CreateUpdateProcedureCommand) procCommand).getUserCommand(); + + if (container != null) { + if (container instanceof StoredProcedure) { + plan.setRequiresTransaction(container.getUpdateCount() > 0); + } + Map params = container.getProcedureParameters(); + plan.setParams(params); + plan.setMetadata(metadata); + if (container instanceof TranslatableProcedureContainer) { + plan.setImplicitParams(((TranslatableProcedureContainer)container).getImplicitParams()); + } + } + + plan.setUpdateProcedure(((CreateUpdateProcedureCommand)procCommand).isUpdateProcedure()); + plan.setOutputElements(((CreateUpdateProcedureCommand)procCommand).getProjectedSymbols()); + + if(debug) { + analysisRecord.println("####################################################"); //$NON-NLS-1$ + analysisRecord.println("PROCEDURE PLAN :"+plan); //$NON-NLS-1$ + analysisRecord.println("####################################################"); //$NON-NLS-1$ + } + + return plan; + } + + /** + * <p> Plan a {@link Block} object, recursively plan each statement in the given block and + * add the resulting {@link ProgramInstruction} a new {@link Program} for the block.</p> + * @param block The <code>Block</code> to be planned + * @param metadata Metadata used during planning + * @param childNodes list of CommandTreeNode objects that contain the ProcessorPlans of the child nodes of this procedure + * @param debug Boolean detemining if procedure plan needs to be printed for debug purposes + * @param analysisRecord TODO + * @return A Program resulting in the block planning + * @throws QueryPlannerException if invalid statement is encountered in the block + * @throws QueryMetadataException if there is an error accessing metadata + * @throws TeiidComponentException if unexpected error occurs + */ + private Program planBlock(CreateUpdateProcedureCommand parentProcCommand, Block block, QueryMetadataInterface metadata, boolean debug, IDGenerator idGenerator, CapabilitiesFinder capFinder, AnalysisRecord analysisRecord, CommandContext context) + throws QueryPlannerException, QueryMetadataException, TeiidComponentException { + + // Generate program and add instructions + // this program represents the block on the procedure + // instruction in the program would correspond to statements in the block + Program programBlock = new Program(); + + // plan each statement in the block + for (Statement statement : block.getStatements()) { + Object instruction = planStatement(parentProcCommand, statement, metadata, debug, idGenerator, capFinder, analysisRecord, context); + if(instruction instanceof ProgramInstruction){ + programBlock.addInstruction((ProgramInstruction)instruction); + }else{ + //an array of ProgramInstruction + ProgramInstruction[] insts = (ProgramInstruction[])instruction; + for(int i=0; i<insts.length; i++){ + programBlock.addInstruction(insts[i]); + } + } + } + + return programBlock; + } + + /** + * <p> Plan a {@link Statement} object, depending on the type of the statement construct the appropriate + * {@link ProgramInstruction} return it to added to a {@link Program}. If the statement references a + * <code>Command</code>, it looks up the child CommandTreeNodes to get approproiate child's ProcessrPlan + * and uses it for constructing the necessary instruction.</p> + * @param statement The statement to be planned + * @param metadata Metadata used during planning + * @param childNodes list of CommandTreeNode objects that contain the ProcessorPlans of the child nodes of this procedure + * @param debug Boolean detemining if procedure plan needs to be printed for debug purposes + * @param analysisRecord TODO + * @return An array containing index of the next child to be accessesd and the ProgramInstruction resulting + * in the statement planning + * @throws QueryPlannerException if invalid statement is encountered + * @throws QueryMetadataException if there is an error accessing metadata + * @throws TeiidComponentException if unexpected error occurs + */ + private Object planStatement(CreateUpdateProcedureCommand parentProcCommand, Statement statement, QueryMetadataInterface metadata, boolean debug, IDGenerator idGenerator, CapabilitiesFinder capFinder, AnalysisRecord analysisRecord, CommandContext context) + throws QueryPlannerException, QueryMetadataException, TeiidComponentException { + + int stmtType = statement.getType(); + // object array containing updated child index and the process instruction + //Object array[] = new Object[2]; + // program instr resulting in planning this statement + Object instruction = null; + switch(stmtType) { + case Statement.TYPE_ASSIGNMENT: + case Statement.TYPE_DECLARE: + { + AssignmentInstruction assignInstr = new AssignmentInstruction(); + instruction = assignInstr; + + AssignmentStatement assignStmt = (AssignmentStatement)statement; + + assignInstr.setVariable(assignStmt.getVariable()); + + if(assignStmt.hasCommand()) { + assignInstr.setExpression(new ScalarSubquery(assignStmt.getCommand())); + } else if (assignStmt.hasExpression()) { + Expression asigExpr = assignStmt.getExpression(); + assignInstr.setExpression(asigExpr); + } + if(debug) { + analysisRecord.println("\tASSIGNMENT\n" + statement); //$NON-NLS-1$ + } + break; + } + case Statement.TYPE_ERROR: + { + ErrorInstruction error = new ErrorInstruction(); + instruction = error; + RaiseErrorStatement res = (RaiseErrorStatement)statement; + + if(res.hasCommand()) { + error.setExpression(new ScalarSubquery(res.getCommand())); + } else if (res.hasExpression()) { + Expression asigExpr = res.getExpression(); + error.setExpression(asigExpr); + } + if(debug) { + analysisRecord.println("\tERROR STATEMENT:\n" + statement); //$NON-NLS-1$ + } + break; + } + case Statement.TYPE_COMMAND: + { + CommandStatement cmdStmt = (CommandStatement) statement; + Command command = cmdStmt.getCommand(); + ProcessorPlan commandPlan = cmdStmt.getCommand().getProcessorPlan(); + + if (command.getType() == Command.TYPE_DYNAMIC){ + instruction = new ExecDynamicSqlInstruction(parentProcCommand,((DynamicCommand)command), metadata, idGenerator, capFinder ); + }else{ + instruction = new CreateCursorResultSetInstruction(CreateCursorResultSetInstruction.RS_NAME, commandPlan); + } + + if(debug) { + analysisRecord.println("\tCOMMAND STATEMENT:\n " + statement); //$NON-NLS-1$ + analysisRecord.println("\t\tSTATEMENT COMMAND PROCESS PLAN:\n " + commandPlan); //$NON-NLS-1$ + } + break; + } + case Statement.TYPE_IF: + { + IfStatement ifStmt = (IfStatement)statement; + Program ifProgram = planBlock(parentProcCommand, ifStmt.getIfBlock(), metadata, debug, idGenerator, capFinder, analysisRecord, context); + Program elseProgram = null; + if(ifStmt.hasElseBlock()) { + elseProgram = planBlock(parentProcCommand, ifStmt.getElseBlock(), metadata, debug, idGenerator, capFinder, analysisRecord, context); + } + instruction = new IfInstruction(ifStmt.getCondition(), ifProgram, elseProgram); + if(debug) { + analysisRecord.println("\tIF STATEMENT:\n" + statement); //$NON-NLS-1$ + } + break; + } + case Statement.TYPE_BREAK: + { + if(debug) { + analysisRecord.println("\tBREAK STATEMENT:\n" + statement); //$NON-NLS-1$ + } + instruction = new BreakInstruction(); + break; + } + case Statement.TYPE_CONTINUE: + { + if(debug) { + analysisRecord.println("\tCONTINUE STATEMENT:\n" + statement); //$NON-NLS-1$ + } + instruction = new ContinueInstruction(); + break; + } + case Statement.TYPE_LOOP: + { + LoopStatement loopStmt = (LoopStatement)statement; + if(debug) { + analysisRecord.println("\tLOOP STATEMENT:\n" + statement); //$NON-NLS-1$ + } + String rsName = loopStmt.getCursorName(); + + ProcessorPlan commandPlan = loopStmt.getCommand().getProcessorPlan(); + + Program loopProgram = planBlock(parentProcCommand, loopStmt.getBlock(), metadata, debug, idGenerator, capFinder, analysisRecord, context); + instruction = new LoopInstruction(loopProgram, rsName, commandPlan); + break; + } + case Statement.TYPE_WHILE: + { + WhileStatement whileStmt = (WhileStatement)statement; + Program whileProgram = planBlock(parentProcCommand, whileStmt.getBlock(), metadata, debug, idGenerator, capFinder, analysisRecord, context); + if(debug) { + analysisRecord.println("\tWHILE STATEMENT:\n" + statement); //$NON-NLS-1$ + } + instruction = new WhileInstruction(whileProgram, whileStmt.getCondition()); + break; + } + default: + throw new QueryPlannerException(QueryExecPlugin.Util.getString("ProcedurePlanner.bad_stmt", stmtType)); //$NON-NLS-1$ + } + return instruction; + } + +} // END CLASS Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -37,7 +37,6 @@ import org.teiid.query.metadata.TempMetadataAdapter; import org.teiid.query.metadata.TempMetadataStore; import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; -import org.teiid.query.optimizer.proc.ProcedurePlanner; import org.teiid.query.optimizer.relational.RelationalPlanner; import org.teiid.query.optimizer.xml.XMLPlanner; import org.teiid.query.processor.ProcessorPlan; @@ -107,7 +106,9 @@ if (command.getType() == Command.TYPE_QUERY && command instanceof Query && QueryResolver.isXMLQuery((Query)command, metadata)) { result = XML_PLANNER.optimize(command, idGenerator, metadata, capFinder, analysisRecord, context); } else { - result = new RelationalPlanner().optimize(command, idGenerator, metadata, capFinder, analysisRecord, context); + RelationalPlanner planner = new RelationalPlanner(); + planner.initialize(command, idGenerator, metadata, capFinder, analysisRecord, context); + result = planner.optimize(command); } } catch (QueryResolverException e) { throw new TeiidRuntimeException(e); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -35,6 +35,7 @@ import org.teiid.api.exception.query.QueryParserException; import org.teiid.api.exception.query.QueryPlannerException; import org.teiid.api.exception.query.QueryResolverException; +import org.teiid.api.exception.query.QueryValidatorException; import org.teiid.client.plan.Annotation; import org.teiid.client.plan.Annotation.Priority; import org.teiid.core.TeiidComponentException; @@ -50,7 +51,7 @@ import org.teiid.query.metadata.QueryMetadataInterface; import org.teiid.query.metadata.TempMetadataAdapter; import org.teiid.query.metadata.TempMetadataID; -import org.teiid.query.optimizer.CommandPlanner; +import org.teiid.query.metadata.TempMetadataStore; import org.teiid.query.optimizer.QueryOptimizer; import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; import org.teiid.query.optimizer.relational.plantree.NodeConstants; @@ -112,7 +113,7 @@ * to produce a * {@link org.teiid.query.processor.relational.RelationalPlan RelationalPlan}. */ -public class RelationalPlanner implements CommandPlanner { +public class RelationalPlanner { private AnalysisRecord analysisRecord; private Command parentCommand; @@ -122,18 +123,13 @@ private QueryMetadataInterface metadata; private PlanHints hints = new PlanHints(); private Option option; - + public ProcessorPlan optimize( - Command command, - IDGenerator idGenerator, - QueryMetadataInterface metadata, - CapabilitiesFinder capFinder, - AnalysisRecord analysisRecord, CommandContext context) + Command command) throws QueryPlannerException, QueryMetadataException, TeiidComponentException { - initialize(command, idGenerator, metadata, capFinder, analysisRecord, context); boolean debug = analysisRecord.recordDebug(); if(debug) { @@ -900,11 +896,12 @@ QueryNode qnode = null; Object metadataID = virtualGroup.getMetadataID(); - boolean noCache = false; + boolean noCache = isNoCacheGroup(metadata, metadataID, option); boolean isMaterializedGroup = metadata.hasMaterialization(metadataID); String cacheString = "select"; //$NON-NLS-1$ + String groupName = metadata.getFullName(metadataID); + if( isMaterializedGroup) { - noCache = isNoCacheGroup(metadata, metadataID, option); if(noCache){ //not use cache qnode = metadata.getVirtualPlan(metadataID); @@ -912,7 +909,6 @@ recordMaterializationTableAnnotation(virtualGroup, analysisRecord, matTableName, "SimpleQueryResolver.materialized_table_not_used"); //$NON-NLS-1$ }else{ // Default query for a materialized group - go to cached table - String groupName = metadata.getFullName(metadataID); String matTableName = metadata.getFullName(metadata.getMaterialization(metadataID)); qnode = new QueryNode(groupName, "SELECT * FROM " + matTableName); //$NON-NLS-1$ cacheString = "matview"; //$NON-NLS-1$ @@ -923,19 +919,74 @@ qnode = metadata.getVirtualPlan(metadataID); } - Command result = (Command)metadata.getFromMetadataCache(virtualGroup.getMetadataID(), "transformation/" + cacheString); //$NON-NLS-1$ + Command result = getCommand(virtualGroup, qnode, cacheString, metadata); + if (!isMaterializedGroup && result.isCache()) { + result = handleCacheHint(virtualGroup, groupName, noCache, result); + } + return QueryRewriter.rewrite(result, metadata, context); + } + + private Command getCommand(GroupSymbol virtualGroup, QueryNode qnode, + String cacheString, QueryMetadataInterface qmi) throws TeiidComponentException, + QueryMetadataException, QueryResolverException, + QueryValidatorException { + Command result = (Command)qmi.getFromMetadataCache(virtualGroup.getMetadataID(), "transformation/" + cacheString); //$NON-NLS-1$ if (result != null) { result = (Command)result.clone(); } else { //parse, resolve, validate - result = convertToSubquery(qnode, metadata); - QueryResolver.resolveCommand(result, Collections.EMPTY_MAP, metadata, analysisRecord); - Request.validateWithVisitor(new ValidationVisitor(), metadata, result); - metadata.addToMetadataCache(virtualGroup.getMetadataID(), "transformation/" + cacheString, result.clone()); //$NON-NLS-1$ - } - return QueryRewriter.rewrite(result, metadata, context); - } + result = convertToSubquery(qnode, qmi); + QueryResolver.resolveCommand(result, Collections.EMPTY_MAP, qmi, analysisRecord); + Request.validateWithVisitor(new ValidationVisitor(), qmi, result); + qmi.addToMetadataCache(virtualGroup.getMetadataID(), "transformation/" + cacheString, result.clone()); //$NON-NLS-1$ + } + return result; + } + private Command handleCacheHint(GroupSymbol virtualGroup, String name, boolean noCache, Command result) + throws TeiidComponentException, QueryMetadataException, QueryResolverException, QueryValidatorException { + TempMetadataStore store = context.getGlobalTableStore().getMetadataStore(); + String matTableName = "#MAT_" + name; //$NON-NLS-1$ + TempMetadataID id = store.getTempGroupID(matTableName); + //define the table preserving the primary key + if (id == null) { + synchronized (store) { + id = store.getTempGroupID(matTableName); + if (id == null) { + //TODO: this could be done with a generated create + id = store.addTempGroup(matTableName, result.getProjectedSymbols(), false, true); + + //primary key + Collection keys = metadata.getUniqueKeysInGroup(virtualGroup.getMetadataID()); + for (Object object : keys) { + if (metadata.isPrimaryKey(object)) { + List cols = metadata.getElementIDsInKey(object); + ArrayList<TempMetadataID> primaryKey = new ArrayList<TempMetadataID>(cols.size()); + for (Object coldId : cols) { + int pos = metadata.getPosition(coldId) - 1; + primaryKey.add(id.getElements().get(pos)); + } + break; + } + } + //version column? + + //add timestamp? + } + } + } + if (noCache) { + //TODO: update the table + return result; + } +/* QueryMetadataInterface qmi = new TempMetadataAdapter(metadata, context.getGlobalTableStore().getMetadataStore()); + QueryNode qn = new QueryNode(virtualGroup.getName(), "select * from " + matTableName); //$NON-NLS-1$ + Query query = (Query)getCommand(virtualGroup, qn, "matview", qmi); //$NON-NLS-1$ + query.getFrom().getGroups().get(0).setGlobalTable(true); + return query;*/ + return result; + } + public static boolean isNoCacheGroup(QueryMetadataInterface metadata, Object metadataID, Option option) throws QueryMetadataException, Copied: trunk/engine/src/main/java/org/teiid/query/processor/BatchedUpdatePlan.java (from rev 2390, trunk/engine/src/main/java/org/teiid/query/processor/batch/BatchedUpdatePlan.java) =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/BatchedUpdatePlan.java (rev 0) +++ trunk/engine/src/main/java/org/teiid/query/processor/BatchedUpdatePlan.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -0,0 +1,241 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.processor; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.teiid.client.plan.PlanNode; +import org.teiid.common.buffer.BlockedException; +import org.teiid.common.buffer.BufferManager; +import org.teiid.common.buffer.TupleBatch; +import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidProcessingException; +import org.teiid.query.sql.lang.Command; +import org.teiid.query.sql.util.VariableContext; +import org.teiid.query.util.CommandContext; + + + +/** + * Plan for execution for a batched update command. The plan executes the child plans of the + * individual commands in order. + * + * If variableContexts are provided, then this is a bulk update where all plans are the same object. + * + * @since 4.2 + */ +public class BatchedUpdatePlan extends ProcessorPlan { + + /** Array that holds the child update plans */ + private ProcessorPlan[] updatePlans; + /** */ + private boolean[] planOpened; + /** Array that holds the update counts for each command in this batch */ + private List[] updateCounts; + /** The position of the plan currently being executed */ + private int planIndex = 0; + /** The position of the command for which the update count is being retrieved */ + private int commandIndex = 0; + + private List<VariableContext> contexts; //only set for bulk updates + + /** + * + * @param childPlans the child update plans for this batch + * @param commandsInBatch The total number of commands in this batch. This does not always equal the number of plans if some + * commands have been batched together. + * @since 4.2 + */ + public BatchedUpdatePlan(List childPlans, int commandsInBatch, List<VariableContext> contexts) { + this.updatePlans = (ProcessorPlan[])childPlans.toArray(new ProcessorPlan[childPlans.size()]); + this.planOpened = new boolean[updatePlans.length]; + this.updateCounts = new List[commandsInBatch]; + this.contexts = contexts; + } + + /** + * @see java.lang.Object#clone() + * @since 4.2 + */ + public BatchedUpdatePlan clone() { + List<ProcessorPlan> clonedPlans = new ArrayList<ProcessorPlan>(updatePlans.length); + + clonedPlans.add(updatePlans[0].clone()); + for (int i = 1; i <updatePlans.length; i++) { + if (contexts == null) { + clonedPlans.add(updatePlans[1].clone()); + } else { + clonedPlans.add(clonedPlans.get(0)); + } + } + return new BatchedUpdatePlan(clonedPlans, updateCounts.length, contexts); + } + + /** + * @see org.teiid.query.processor.ProcessorPlan#initialize(org.teiid.query.util.CommandContext, org.teiid.query.processor.ProcessorDataManager, org.teiid.common.buffer.BufferManager) + * @since 4.2 + */ + public void initialize(CommandContext context, ProcessorDataManager dataMgr, BufferManager bufferMgr) { + context = context.clone(); + context.setVariableContext(new VariableContext()); //start a new root variable context + this.setContext(context); + // Initialize all the child plans + for (int i = 0; i < getPlanCount(); i++) { + updatePlans[i].initialize(context, dataMgr, bufferMgr); + } + } + + /** + * @see org.teiid.query.processor.ProcessorPlan#getOutputElements() + * @since 4.2 + */ + public List getOutputElements() { + return Command.getUpdateCommandSymbol(); + } + + /** + * @see org.teiid.query.processor.ProcessorPlan#open() + * @since 4.2 + */ + public void open() throws TeiidComponentException, TeiidProcessingException { + // It's ok to open() the first plan, as it is not dependent on any prior commands. + // See note for defect 16166 in the nextBatch() method. + openPlan(); + } + + /** + * @see org.teiid.query.processor.ProcessorPlan#nextBatch() + * @since 4.2 + */ + public TupleBatch nextBatch() throws BlockedException, TeiidComponentException, TeiidProcessingException { + for (;planIndex < updatePlans.length; planIndex++) { + if (!planOpened[planIndex]) { // Open the plan only once + /* Defect 16166 + * Some commands in a batch may depend on updates by previous commands in the same batch. A call + * to open() usually submits an atomic command, so calling open() on all the child plans at the same time + * will mean that the datasource may not be in the state expected by a later command within the batch. So, + * for a batch of commands, we only open() a later plan when we are finished with the previous plan to + * guarantee that the commands in the previous plan are completed before the commands in any subsequent + * plans are executed. + */ + openPlan(); + } + // Execute nextBatch() on each plan in sequence + List<List> currentBatch = updatePlans[planIndex].nextBatch().getTuples(); // Can throw BlockedException + for (int i = 0; i < currentBatch.size(); i++, commandIndex++) { + updateCounts[commandIndex] = currentBatch.get(i); + } + + // since we are done with the plan explicitly close it. + updatePlans[planIndex].close(); + } + // Add tuples to current batch + TupleBatch batch = new TupleBatch(1, updateCounts); + batch.setTerminationFlag(true); + return batch; + } + + private void openPlan() throws TeiidComponentException, + TeiidProcessingException { + if (this.contexts != null && !this.contexts.isEmpty()) { + CommandContext context = updatePlans[planIndex].getContext(); + context.getVariableContext().clear(); + VariableContext currentValues = this.contexts.get(planIndex); + context.getVariableContext().putAll(currentValues); + } + updatePlans[planIndex].reset(); + updatePlans[planIndex].open(); + planOpened[planIndex] = true; + } + + /** + * @see org.teiid.query.processor.ProcessorPlan#close() + * @since 4.2 + */ + public void close() throws TeiidComponentException { + // if the plan opened but the atomic request got cancelled then close the last plan node. + if (planIndex < updatePlans.length && planOpened[planIndex]) { + updatePlans[planIndex].close(); + } + } + + + /** + * @see org.teiid.query.processor.ProcessorPlan#reset() + * @since 4.2 + */ + public void reset() { + super.reset(); + for (int i = 0; i < updatePlans.length; i++) { + updatePlans[i].reset(); + planOpened[i] = false; + } + for (int i = 0; i < updateCounts.length; i++) { + updateCounts[i] = null; + } + planIndex = 0; + commandIndex = 0; + } + + public PlanNode getDescriptionProperties() { + PlanNode props = super.getDescriptionProperties(); + for (int i = 0; i < getPlanCount(); i++) { + props.addProperty("Batch Plan " + i, updatePlans[i].getDescriptionProperties()); //$NON-NLS-1$ + } + return props; + } + + public String toString() { + StringBuffer val = new StringBuffer("BatchedUpdatePlan {\n"); //$NON-NLS-1$ + for (int i = 0; i < getPlanCount(); i++) { + val.append(updatePlans[i]) + .append("\n"); //$NON-NLS-1$ + } + val.append("}\n"); //$NON-NLS-1$ + return val.toString(); + } + + private int getPlanCount() { + return (contexts != null?1:updatePlans.length); + } + + /** + * Returns the child plans for this batch. Used primarily for unit tests. + * @return + * @since 4.2 + */ + public List getUpdatePlans() { + return Arrays.asList(updatePlans); + } + + @Override + public boolean requiresTransaction(boolean transactionalReads) { + if (updatePlans.length == 1) { + return updatePlans[0].requiresTransaction(transactionalReads); + } + return true; + } + +} Modified: trunk/engine/src/main/java/org/teiid/query/processor/ProcessorDataManager.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/ProcessorDataManager.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/ProcessorDataManager.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -32,7 +32,7 @@ public interface ProcessorDataManager { - TupleSource registerRequest(Object processorId, Command command, String modelName, String connectorBindingId, int nodeID) + TupleSource registerRequest(CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException; /** Modified: trunk/engine/src/main/java/org/teiid/query/processor/TempTableDataManager.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/TempTableDataManager.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/TempTableDataManager.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -32,10 +32,7 @@ /** - * This proxy ProcessorDataManager is used during XML query processing to handle temporary groups - * in the document model. Temp groups are materialized during processing, and their tuple sources - * are cached, so this proxy shortcuts the need to go to the DataTierManager and immediately - * returns the tuple source (synchronously) if a temp group's source is what's being requested. + * This proxy ProcessorDataManager is used to handle temporary tables. */ public class TempTableDataManager implements ProcessorDataManager { @@ -45,38 +42,28 @@ /** * Constructor takes the "real" ProcessorDataManager that this object will be a proxy to, * and will pass most calls through to transparently. Only when a request is registered for - * a temp group will this proxy do it's thing. A ProcessorEnvironment is needed to to - * access cached information about temp groups + * a temp group will this proxy do it's thing. * @param processorDataManager the real ProcessorDataManager that this object is a proxy to - * @param env a ProcessorEnvironment implementation */ public TempTableDataManager(ProcessorDataManager processorDataManager, TempTableStore tempTableStore){ this.processorDataManager = processorDataManager; this.tempTableStore = tempTableStore; } - /** - * This is the magic method. If the command is selecting from a temporary group, that - * temporary groups tuple source (which is cached in the ProcessorEnvironment) will - * be retrieved and immediately (synchronously) delivered to the QueryProcessor. - * If a temp group is <i>not</i> being selected from, then this request will be - * passed through to the underlying ProcessorDataManager. - * @throws TeiidProcessingException - */ public TupleSource registerRequest( - Object processorID, + CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException { if(tempTableStore != null) { - TupleSource result = tempTableStore.registerRequest(command); + TupleSource result = tempTableStore.registerRequest(context, command); if (result != null) { return result; } } - return this.processorDataManager.registerRequest(processorID, command, modelName, connectorBindingId, nodeID); + return this.processorDataManager.registerRequest(context, command, modelName, connectorBindingId, nodeID); } public Object lookupCodeValue( Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/AssignmentInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/AssignmentInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/AssignmentInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -33,7 +33,6 @@ import org.teiid.logging.LogConstants; import org.teiid.logging.LogManager; import org.teiid.query.analysis.AnalysisRecord; -import org.teiid.query.processor.program.ProgramInstruction; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.util.VariableContext; @@ -120,7 +119,7 @@ } /** - * @see org.teiid.query.processor.program.ProgramInstruction#clone() + * @see org.teiid.query.processor.proc.ProgramInstruction#clone() */ public AssignmentInstruction clone() { AssignmentInstruction clone = new AssignmentInstruction(); Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/BreakInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/BreakInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/BreakInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -26,7 +26,6 @@ import org.teiid.client.plan.PlanNode; import org.teiid.core.TeiidComponentException; -import org.teiid.query.processor.program.ProgramInstruction; /** Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/CreateCursorResultSetInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/CreateCursorResultSetInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/CreateCursorResultSetInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -31,7 +31,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.query.processor.ProcessorPlan; -import org.teiid.query.processor.program.ProgramInstruction; /** Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/ErrorInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/ErrorInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/ErrorInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -29,7 +29,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.logging.LogManager; -import org.teiid.query.processor.program.ProgramInstruction; import org.teiid.query.sql.symbol.Expression; @@ -54,7 +53,7 @@ } /** - * @see org.teiid.query.processor.program.ProgramInstruction#clone() + * @see org.teiid.query.processor.proc.ProgramInstruction#clone() */ public ErrorInstruction clone() { ErrorInstruction clone = new ErrorInstruction(); Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/ExecDynamicSqlInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/ExecDynamicSqlInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/ExecDynamicSqlInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -49,8 +49,6 @@ import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; import org.teiid.query.parser.QueryParser; import org.teiid.query.processor.ProcessorPlan; -import org.teiid.query.processor.program.Program; -import org.teiid.query.processor.program.ProgramInstruction; import org.teiid.query.resolver.QueryResolver; import org.teiid.query.resolver.util.ResolveVirtualGroupCriteriaVisitor; import org.teiid.query.rewriter.QueryRewriter; Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/IfInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/IfInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/IfInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -29,8 +29,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.logging.LogManager; -import org.teiid.query.processor.program.Program; -import org.teiid.query.processor.program.ProgramInstruction; import org.teiid.query.sql.lang.Criteria; Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/LoopInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/LoopInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/LoopInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -33,7 +33,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.query.processor.ProcessorPlan; -import org.teiid.query.processor.program.Program; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.util.VariableContext; Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -55,8 +55,6 @@ import org.teiid.query.processor.ProcessorPlan; import org.teiid.query.processor.QueryProcessor; import org.teiid.query.processor.TempTableDataManager; -import org.teiid.query.processor.program.Program; -import org.teiid.query.processor.program.ProgramInstruction; import org.teiid.query.processor.relational.SubqueryAwareEvaluator; import org.teiid.query.sql.ProcedureReservedWords; import org.teiid.query.sql.lang.Criteria; @@ -204,7 +202,7 @@ context.setValue(entry.getKey(), value); } } - tempTableStore = new TempTableStore(bufferMgr, getContext().getConnectionID(), null); + tempTableStore = new TempTableStore(bufferMgr, getContext().getConnectionID()); this.dataMgr = new TempTableDataManager(dataMgr, tempTableStore); } this.evaluatedParams = true; @@ -422,7 +420,7 @@ //this may not be the first time the plan is being run command.reset(); - CommandContext subContext = (CommandContext) getContext().clone(); + CommandContext subContext = getContext().clone(); subContext.setVariableContext(this.currentVarContext); subContext.setTempTableStore(getTempTableStore()); state = new CursorState(); Copied: trunk/engine/src/main/java/org/teiid/query/processor/proc/Program.java (from rev 2390, trunk/engine/src/main/java/org/teiid/query/processor/program/Program.java) =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/Program.java (rev 0) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/Program.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -0,0 +1,216 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.processor.proc; + +import java.util.ArrayList; +import java.util.List; + +import org.teiid.client.plan.PlanNode; + + +/** + * A program is a sequence of {@link ProgramInstruction ProgramInstruction}. Certain + * ProgramInstructions, such as {@link IfInstruction} and {@link WhileInstruction} may + * have pointers to sub programs. + */ +public class Program implements Cloneable { + + private List<ProgramInstruction> programInstructions; + private int counter = 0; + + /** + * Constructor for Program. + */ + public Program() { + super(); + } + + /** + * Returns the next instruction to be executed, or null if there are + * none or no more instructions. + * @return ProgramInstruction to be executed next, or null if there + * are no more to execute (or if this Program is empty) + */ + public ProgramInstruction getCurrentInstruction(){ + return getInstructionAtIndex(counter); + } + + /** + * Increments the program counter, so that the next call to + * {@link #getCurrentInstruction} will return the following + * instruction. This method is intended to be used by a + * ProcessingInstruction itself, to control the flow of execution. + */ + public void incrementProgramCounter(){ + counter++; + } + + /** + * Decrements the program counter, so that the next call to + * {@link #getCurrentInstruction} will return the previous + * instruction. This method is intended to be used by a + * ProcessingInstruction itself, to control the flow of execution. + */ + public void decrementProgramCounter(){ + counter--; + } + + /** + * Resets this program, so it can be run through again. + */ + public void resetProgramCounter(){ + counter = 0; + } + + int getProgramCounter(){ + return counter; + } + + + /** + * Returns the instruction to be executed at the indicated index, + * or null if there is no instruction at that index. + * @return instruction to be executed at the indicated index, + * or null if there is no instruction at that index. + */ + public ProgramInstruction getInstructionAt(int instructionIndex){ + return getInstructionAtIndex(instructionIndex); + } + + + public void addInstruction(ProgramInstruction instruction){ + if (instruction != null){ + getProcessorInstructions().add(instruction); + } + } + + public void addInstructions(Program instructions){ + if (instructions != null){ + getProcessorInstructions().addAll(instructions.getProcessorInstructions()); + } + } + + /** + * Produces a deep clone. + */ + public Object clone(){ + Program program = new Program(); + program.counter = this.counter; + + if (this.programInstructions != null){ + ArrayList<ProgramInstruction> clonedInstructions = new ArrayList<ProgramInstruction>(this.programInstructions.size()); + for (ProgramInstruction pi : this.programInstructions) { + clonedInstructions.add( pi.clone() ); + } + program.programInstructions = clonedInstructions; + } + + return program; + } + + public PlanNode getDescriptionProperties() { + PlanNode props = new PlanNode("Program"); //$NON-NLS-1$ + + if(this.programInstructions != null) { + for (int i = 0; i < programInstructions.size(); i++) { + ProgramInstruction inst = programInstructions.get(i); + PlanNode childProps = inst.getDescriptionProperties(); + props.addProperty("Instruction " + i, childProps); //$NON-NLS-1$ + } + } + return props; + } + + //========================================================================= + //UTILITY + //========================================================================= + + /** + * Returns the instruction to be executed at the indicated index, + * or null if there is no instruction at that index. + * @return instruction to be executed at the indicated index, + * or null if there is no instruction at that index. + */ + private ProgramInstruction getInstructionAtIndex(int instructionIndex){ + if (programInstructions != null){ + if (instructionIndex < getProcessorInstructions().size()){ + return getProcessorInstructions().get(instructionIndex); + } + } + return null; + } + + public List<ProgramInstruction> getProcessorInstructions(){ + if (programInstructions == null){ + programInstructions = new ArrayList<ProgramInstruction>(); + } + return programInstructions; + } + + public String toString() { + StringBuilder str = new StringBuilder(); + + programToString(str); + + return "PROGRAM counter " + this.counter + "\n" + str.toString(); //$NON-NLS-1$ //$NON-NLS-2$ + } + + /** + * This method calls itself recursively if either a While or If instruction is encountered. + * The sub program(s) from those kinds of instructions are passed, recursively, into this + * method. + */ + private final void programToString(StringBuilder str) { + + int instructionIndex = 0; + ProgramInstruction inst = getInstructionAt(instructionIndex); + + while(inst != null) { + + printLine(instructionIndex++, inst.toString(), str); + + if(instructionIndex > 1000) { + printLine(instructionIndex, "[OUTPUT TRUNCATED...]", str); //$NON-NLS-1$ + break; + } + + inst = getInstructionAt(instructionIndex); + } + } + + private static final void printLine(int counter, String line, StringBuilder buffer) { + // Pad counter with spaces + String counterStr = "" + counter + ": "; //$NON-NLS-1$ //$NON-NLS-2$ + if(counter < 10) { + counterStr += " "; //$NON-NLS-1$ + } + if(counterStr.length() == 1) { + counterStr += " "; //$NON-NLS-1$ + } else if(counterStr.length() == 2) { + counterStr += " "; //$NON-NLS-1$ + } + + buffer.append(counterStr + line + "\n"); //$NON-NLS-1$ + } + +} Copied: trunk/engine/src/main/java/org/teiid/query/processor/proc/ProgramInstruction.java (from rev 2390, trunk/engine/src/main/java/org/teiid/query/processor/program/ProgramInstruction.java) =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/ProgramInstruction.java (rev 0) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/ProgramInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -0,0 +1,85 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.processor.proc; + +import java.util.List; + +import org.teiid.client.plan.PlanNode; +import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidProcessingException; +import org.teiid.query.processor.ProcessorPlan; + + +/** + * <p>Abstract superclass of all program instructions.</p> + * + * <p>All processor + * instructions need to be cloneable, but it most cases the default + * Object.clone operation will suffice, since in most cases the processing + * instructions will be stateless, or the state will be immutable. + * The exception to this are instructions that have sub programs in them - + * those sub programs need to be cloned.</p> + */ +public abstract class ProgramInstruction implements Cloneable { + + public ProgramInstruction() { + } + + /** + * Allow this instruction to do whatever processing it needs, and to + * in turn manipulate the running program. A typical instruction should simply {@link + * Program#incrementProgramCounter increment} the program counter of the current program, but specialized + * instructions may add sub programs to the stack or not increment the counter (so that they are executed again.) + */ + public abstract void process(ProcedurePlan env) + throws TeiidComponentException, TeiidProcessingException; + + /** + * Finds all nested plans and returns them. + * @return List of ProcessorPlan + * @since 4.2 + */ + public List<ProcessorPlan> getChildPlans() { + return null; + } + + /** + * Override Object.clone() to make the method public. This method + * simply calls super.clone(), deferring to the default shallow + * cloning. Some ProcessorInstruction subclasses may need to + * override with custom safe or deep cloning. + * @return shallow clone + */ + public ProgramInstruction clone() { + try { + return (ProgramInstruction)super.clone(); + } catch (CloneNotSupportedException e) { + //should never get here, since + //this Class does support clone + } + return null; + } + + public abstract PlanNode getDescriptionProperties(); + +} Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/RepeatedInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/RepeatedInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/RepeatedInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -24,7 +24,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; -import org.teiid.query.processor.program.Program; public interface RepeatedInstruction { Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/WhileInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/WhileInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/WhileInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -29,8 +29,6 @@ import org.teiid.client.plan.PlanNode; import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; -import org.teiid.query.processor.program.Program; -import org.teiid.query.processor.program.ProgramInstruction; import org.teiid.query.sql.lang.Criteria; Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/AccessNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/AccessNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/AccessNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -173,7 +173,7 @@ private void registerRequest(Command atomicCommand) throws TeiidComponentException, TeiidProcessingException { - tupleSource = getDataManager().registerRequest(this.getContext().getProcessorID(), atomicCommand, modelName, connectorBindingId, getID()); + tupleSource = getDataManager().registerRequest(getContext(), atomicCommand, modelName, connectorBindingId, getID()); } protected boolean processCommandsIndividually() { Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/BatchedUpdateNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/BatchedUpdateNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/BatchedUpdateNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -92,7 +92,7 @@ Command updateCommand = (Command)updateCommands.get(i).clone(); CommandContext context = this.getContext(); if (this.contexts != null && !this.contexts.isEmpty()) { - context = (CommandContext)context.clone(); + context = context.clone(); context.setVariableContext(this.contexts.get(i)); } boolean needProcessing = false; @@ -110,7 +110,7 @@ } if (!commandsToExecute.isEmpty()) { BatchedUpdateCommand command = new BatchedUpdateCommand(commandsToExecute); - tupleSource = getDataManager().registerRequest(this.getContext().getProcessorID(), command, modelName, null, getID()); + tupleSource = getDataManager().registerRequest(getContext(), command, modelName, null, getID()); } } Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentProcedureAccessNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentProcedureAccessNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentProcedureAccessNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -79,7 +79,7 @@ @Override public void open() throws TeiidComponentException, TeiidProcessingException { - CommandContext context = (CommandContext)getContext().clone(); + CommandContext context = getContext().clone(); context.pushVariableContext(new VariableContext()); this.setContext(context); DependentProcedureExecutionNode.shareVariableContext(this, context); Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/PlanExecutionNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/PlanExecutionNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/PlanExecutionNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -69,7 +69,7 @@ throws TeiidComponentException, TeiidProcessingException { super.open(); // Initialize plan for execution - CommandContext subContext = (CommandContext) getContext().clone(); + CommandContext subContext = getContext().clone(); subContext.pushVariableContext(new VariableContext()); plan.initialize(subContext, getDataManager(), this.getBufferManager()); Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -224,7 +224,7 @@ } private void registerRequest(Command command) throws TeiidComponentException, TeiidProcessingException { - tupleSource = getDataManager().registerRequest(this.getContext().getProcessorID(), command, this.modelName, null, getID()); + tupleSource = getDataManager().registerRequest(getContext(), command, this.modelName, null, getID()); } private void closeRequest() { Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/SubqueryAwareEvaluator.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/SubqueryAwareEvaluator.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/SubqueryAwareEvaluator.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -112,7 +112,7 @@ } if (!state.done) { if (state.processor == null) { - CommandContext subContext = (CommandContext) context.clone(); + CommandContext subContext = context.clone(); state.plan.reset(); state.processor = new QueryProcessor(state.plan, subContext, manager, this.dataMgr); if (container.getCommand().getCorrelatedReferences() != null) { Modified: trunk/engine/src/main/java/org/teiid/query/processor/xml/InitializeDocumentInstruction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/xml/InitializeDocumentInstruction.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/xml/InitializeDocumentInstruction.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -30,7 +30,6 @@ import org.teiid.core.TeiidProcessingException; import org.teiid.logging.LogManager; import org.teiid.query.mapping.xml.MappingNodeConstants; -import org.teiid.query.util.XMLFormatConstants; /** @@ -93,9 +92,9 @@ boolean formatted = this.isFormatted; String xmlFormat = env.getXMLFormat(); if(xmlFormat != null) { - if(xmlFormat.equalsIgnoreCase(XMLFormatConstants.XML_TREE_FORMAT)) { + if(xmlFormat.equalsIgnoreCase(XMLPlan.XML_TREE_FORMAT)) { formatted = true; - } else if (xmlFormat.equalsIgnoreCase(XMLFormatConstants.XML_COMPACT_FORMAT)) { + } else if (xmlFormat.equalsIgnoreCase(XMLPlan.XML_COMPACT_FORMAT)) { formatted = false; } } Modified: trunk/engine/src/main/java/org/teiid/query/processor/xml/RelationalPlanExecutor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/xml/RelationalPlanExecutor.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/xml/RelationalPlanExecutor.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -72,7 +72,7 @@ this.bufferMgr = bufferMgr; ProcessorPlan plan = resultInfo.getPlan(); - CommandContext subContext = (CommandContext)context.clone(); + CommandContext subContext = context.clone(); subContext.pushVariableContext(new VariableContext()); this.internalProcessor = new QueryProcessor(plan, subContext, bufferMgr, dataMgr); } Modified: trunk/engine/src/main/java/org/teiid/query/processor/xml/XMLPlan.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/xml/XMLPlan.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/processor/xml/XMLPlan.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -47,8 +47,8 @@ import org.teiid.common.buffer.BufferManager; import org.teiid.common.buffer.TupleBatch; import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidException; import org.teiid.core.TeiidProcessingException; -import org.teiid.core.TeiidException; import org.teiid.core.types.DataTypeManager; import org.teiid.core.types.XMLType; import org.teiid.logging.LogConstants; @@ -61,7 +61,6 @@ import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.tempdata.TempTableStore; -import org.teiid.query.tempdata.TempTableStore; import org.teiid.query.util.CommandContext; import org.teiid.query.util.ErrorMessageKeys; import org.xml.sax.Attributes; @@ -94,6 +93,10 @@ // Post-processing private Collection<SQLXML> xmlSchemas; + /** XML results format: XML results displayed as a tree*/ + public static final String XML_TREE_FORMAT = "Tree"; //$NON-NLS-1$ + /** XML results format: XML results displayed in compact form*/ + public static final String XML_COMPACT_FORMAT = "Compact"; //$NON-NLS-1$ /** * Constructor for XMLPlan. @@ -108,8 +111,8 @@ */ public void initialize(CommandContext context, ProcessorDataManager dataMgr, BufferManager bufferMgr) { setContext(context); - TempTableStore tempTableStore = new TempTableStore(bufferMgr, context.getConnectionID(), (TempTableStore)context.getTempTableStore()); - //this.dataMgr = new StagingTableDataManager(new TempTableDataManager(dataMgr, tempTableStore), env); + TempTableStore tempTableStore = new TempTableStore(bufferMgr, context.getConnectionID()); + tempTableStore.setParentTempTableStore(context.getTempTableStore()); this.dataMgr = new TempTableDataManager(dataMgr, tempTableStore); this.bufferMgr = bufferMgr; this.env.initialize(context, this.dataMgr, this.bufferMgr); Modified: trunk/engine/src/main/java/org/teiid/query/sql/lang/From.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/lang/From.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/sql/lang/From.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -107,11 +107,10 @@ * Adds a new collection of groups to the list * @param groups Collection of {@link GroupSymbol} */ - public void addGroups( Collection groups ) { + public void addGroups( Collection<GroupSymbol> groups ) { if(groups != null) { - Iterator iter = groups.iterator(); - while(iter.hasNext()) { - clauses.add(new UnaryFromClause((GroupSymbol) iter.next())); + for (GroupSymbol groupSymbol : groups) { + clauses.add(new UnaryFromClause(groupSymbol)); } } } @@ -120,8 +119,8 @@ * Returns an ordered list of the groups in all sub-clauses. * @return List of {@link GroupSymbol} */ - public List getGroups() { - List groups = new ArrayList(); + public List<GroupSymbol> getGroups() { + List<GroupSymbol> groups = new ArrayList<GroupSymbol>(); if(clauses != null) { for(int i=0; i<clauses.size(); i++) { FromClause clause = (FromClause) clauses.get(i); Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/GroupSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/GroupSymbol.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/GroupSymbol.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -38,8 +38,8 @@ * first would have name=G, definition=Group1 and the second would have * name=Group2, definition=null.</p> */ -public class GroupSymbol extends Symbol implements Comparable { - +public class GroupSymbol extends Symbol implements Comparable<GroupSymbol> { + public static final String TEMP_GROUP_PREFIX = "#"; //$NON-NLS-1$ /** Definition of the symbol, may be null */ @@ -49,7 +49,7 @@ private Object metadataID; private boolean isTempTable; - + private boolean isGlobalTable; private boolean isProcedure; private String outputDefinition; @@ -152,6 +152,8 @@ /** * Returns true if this is a symbol for a temporary (implicit or explicit) group + * May return false for explicit temp tables prior to resolving. + * see {@link #isTempTable()} * @return * @since 5.5 */ @@ -160,24 +162,16 @@ } public boolean isImplicitTempGroupSymbol() { - return isTempGroupName(getName()) || isTempGroupName(getDefinition()); + return isTempGroupName(getNonCorrelationName()); } /** - * Compare two groups and give an ordering. This is done with the hashcode of the - * lowercased group name. The order is stable. + * Compare two groups and give an ordering. * @param other Other group * @return -1, 0, or 1 depending on how this compares to group */ - public int compareTo(Object other) { - int diff = other.hashCode() - this.hashCode(); - if(diff == 0) { - return 0; - } else if(diff <0) { - return -1; - } else { - return 1; - } + public int compareTo(GroupSymbol o) { + return getCanonicalName().compareTo(o.getCanonicalName()); } /** @@ -193,6 +187,7 @@ copy.setProcedure(isProcedure); copy.setOutputDefinition(this.getOutputDefinition()); copy.setOutputName(this.getOutputName()); + copy.isGlobalTable = isGlobalTable; return copy; } @@ -206,7 +201,7 @@ return true; } - if(obj == null || ! (obj instanceof GroupSymbol)) { + if(!(obj instanceof GroupSymbol)) { return false; } GroupSymbol other = (GroupSymbol) obj; @@ -237,6 +232,11 @@ return name.startsWith(TEMP_GROUP_PREFIX); } + /** + * Returns if this is a Temp Table + * Set after resolving. + * @return + */ public boolean isTempTable() { return this.isTempTable; } @@ -256,4 +256,12 @@ public void setOutputDefinition(String outputDefinition) { this.outputDefinition = outputDefinition; } + + public boolean isGlobalTable() { + return isGlobalTable; + } + + public void setGlobalTable(boolean isGlobalTable) { + this.isGlobalTable = isGlobalTable; + } } Modified: trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -29,6 +29,7 @@ import java.util.Map; import java.util.Set; import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.locks.ReentrantReadWriteLock; import org.teiid.api.exception.query.ExpressionEvaluationException; import org.teiid.common.buffer.BlockedException; @@ -43,6 +44,8 @@ import org.teiid.core.TeiidException; import org.teiid.core.TeiidProcessingException; import org.teiid.core.types.DataTypeManager; +import org.teiid.logging.LogConstants; +import org.teiid.logging.LogManager; import org.teiid.query.QueryPlugin; import org.teiid.query.eval.Evaluator; import org.teiid.query.metadata.TempMetadataID; @@ -124,7 +127,8 @@ this.eval = new Evaluator(map, null, null); this.condition = condition; this.project = shouldProject(); - reserved = reserveBuffers(); + this.reserved = reserveBuffers(); + lock.readLock().lock(); } @Override @@ -152,6 +156,7 @@ @Override public void closeSource() { + lock.readLock().unlock(); bm.releaseBuffers(reserved); reserved = 0; browser.closeSource(); @@ -191,6 +196,8 @@ int process() throws ExpressionEvaluationException, TeiidComponentException, TeiidProcessingException { int reserved = reserveBuffers(); + boolean held = lock.writeLock().isHeldByCurrentThread(); + lock.writeLock().lock(); boolean success = false; try { while (currentTuple != null || (currentTuple = ts.nextTuple()) != null) { @@ -207,18 +214,22 @@ success = true; } finally { bm.releaseBuffers(reserved); - if (!success) { - TupleSource undoTs = undoLog.createIndexedTupleSource(); - List<?> tuple = null; - try { + try { + if (!success) { + TupleSource undoTs = undoLog.createIndexedTupleSource(); + List<?> tuple = null; while ((tuple = undoTs.nextTuple()) != null) { undo(tuple); } - } catch (TeiidException e) { - } + } catch (TeiidException e) { + LogManager.logError(LogConstants.CTX_DQP, e, e.getMessage()); + } finally { + if (!held) { + lock.writeLock().unlock(); + } + close(); } - close(); } return updateCount; } @@ -243,6 +254,7 @@ private BufferManager bm; private String sessionID; private TempMetadataID tid; + private ReentrantReadWriteLock lock = new ReentrantReadWriteLock(); private int keyBatchSize; private int leafBatchSize; @@ -255,9 +267,9 @@ id.setType(DataTypeManager.DefaultDataClasses.INTEGER); columns.add(0, id); rowId = new AtomicInteger(); - tree = bm.createSTree(columns, sessionID, TupleSourceType.PROCESSOR, 1); + tree = bm.createSTree(columns, sessionID, 1); } else { - tree = bm.createSTree(columns, sessionID, TupleSourceType.PROCESSOR, primaryKeyLength); + tree = bm.createSTree(columns, sessionID, primaryKeyLength); } this.columns = columns; this.sessionID = sessionID; Modified: trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -24,11 +24,11 @@ import java.util.ArrayList; import java.util.Arrays; -import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Map; import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; import org.teiid.api.exception.query.QueryProcessingException; import org.teiid.common.buffer.BufferManager; @@ -55,6 +55,7 @@ import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.visitor.ExpressionMappingVisitor; +import org.teiid.query.util.CommandContext; /** * @since 5.5 @@ -62,25 +63,21 @@ public class TempTableStore { private BufferManager buffer; - private TempMetadataStore tempMetadataStore = new TempMetadataStore(); - private Map<String, TempTable> groupToTupleSourceID = new HashMap<String, TempTable>(); + private TempMetadataStore tempMetadataStore = new TempMetadataStore(new ConcurrentHashMap<String, TempMetadataID>()); + private Map<String, TempTable> groupToTupleSourceID = new ConcurrentHashMap<String, TempTable>(); private String sessionID; private TempTableStore parentTempTableStore; - public TempTableStore(BufferManager buffer, String sessionID, TempTableStore parentTempTableStore) { + public TempTableStore(BufferManager buffer, String sessionID) { this.buffer = buffer; this.sessionID = sessionID; - this.parentTempTableStore = parentTempTableStore; } + + public void setParentTempTableStore(TempTableStore parentTempTableStore) { + this.parentTempTableStore = parentTempTableStore; + } - void addTempTable(Create create, boolean removeExistingTable) throws QueryProcessingException{ - String tempTableName = create.getTable().getName().toUpperCase(); - if(tempMetadataStore.getTempGroupID(tempTableName) != null) { - if(!removeExistingTable) { - throw new QueryProcessingException(QueryExecPlugin.Util.getString("TempTableStore.table_exist_error", tempTableName));//$NON-NLS-1$ - } - removeTempTableByName(tempTableName); - } + void addTempTable(String tempTableName, Create create) { List<ElementSymbol> columns = create.getColumns(); //add metadata @@ -109,15 +106,15 @@ return tempMetadataStore; } - public TupleSource registerRequest(Command command) throws TeiidComponentException, TeiidProcessingException{ + public TupleSource registerRequest(CommandContext context, Command command) throws TeiidComponentException, TeiidProcessingException{ if (command instanceof Query) { Query query = (Query)command; - GroupSymbol group = (GroupSymbol)query.getFrom().getGroups().get(0); + GroupSymbol group = query.getFrom().getGroups().get(0); if (!group.isTempGroupSymbol()) { return null; } final String tableName = group.getNonCorrelationName().toUpperCase(); - TempTable table = getTempTable(tableName, command); + TempTable table = getOrCreateTempTable(tableName, command, true); //convert to the actual table symbols (this is typically handled by the languagebridgefactory ExpressionMappingVisitor emv = new ExpressionMappingVisitor(null) { @Override @@ -138,7 +135,7 @@ return null; } final String groupKey = group.getNonCorrelationName().toUpperCase(); - final TempTable table = getTempTable(groupKey, command); + final TempTable table = getOrCreateTempTable(groupKey, command, false); if (command instanceof Insert) { Insert insert = (Insert)command; TupleSource ts = insert.getTupleSource(); @@ -169,11 +166,15 @@ } if (command instanceof Create) { Create create = (Create)command; - addTempTable(create, false); + String tempTableName = create.getTable().getCanonicalName(); + if(tempMetadataStore.getTempGroupID(tempTableName) != null) { + throw new QueryProcessingException(QueryExecPlugin.Util.getString("TempTableStore.table_exist_error", tempTableName));//$NON-NLS-1$ + } + addTempTable(tempTableName, create); return CollectionTupleSource.createUpdateCountTupleSource(0); } if (command instanceof Drop) { - String tempTableName = ((Drop)command).getTable().getName().toUpperCase(); + String tempTableName = ((Drop)command).getTable().getCanonicalName(); removeTempTableByName(tempTableName); return CollectionTupleSource.createUpdateCountTupleSource(0); } @@ -181,17 +182,17 @@ } public void removeTempTables() { - for (String name : new ArrayList<String>( groupToTupleSourceID.keySet() )) { + for (String name : groupToTupleSourceID.keySet()) { removeTempTableByName(name); } } - private TempTable getTempTable(String tempTableID, Command command) throws QueryProcessingException{ + private TempTable getOrCreateTempTable(String tempTableID, Command command, boolean delegate) throws QueryProcessingException{ TempTable tsID = groupToTupleSourceID.get(tempTableID); if(tsID != null) { return tsID; } - if(this.parentTempTableStore != null){ + if(delegate && this.parentTempTableStore != null){ tsID = this.parentTempTableStore.groupToTupleSourceID.get(tempTableID); if(tsID != null) { return tsID; @@ -212,34 +213,9 @@ Create create = new Create(); create.setTable(new GroupSymbol(tempTableID)); create.setColumns(columns); - addTempTable(create, true); + addTempTable(tempTableID, create); return groupToTupleSourceID.get(tempTableID); } - - public boolean hasTempTable(Command command){ - switch (command.getType()) { - case Command.TYPE_INSERT: - { - Insert insert = (Insert)command; - GroupSymbol group = insert.getGroup(); - return group.isTempGroupSymbol(); - } - case Command.TYPE_QUERY: - { - if(command instanceof Query) { - Query query = (Query)command; - GroupSymbol group = (GroupSymbol)query.getFrom().getGroups().get(0); - return group.isTempGroupSymbol(); - } - break; - } - case Command.TYPE_CREATE: - return true; - case Command.TYPE_DROP: - return true; - } - return false; - } public Set<String> getAllTempTables() { return new HashSet<String>(this.groupToTupleSourceID.keySet()); Modified: trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -42,6 +42,7 @@ import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.util.VariableContext; +import org.teiid.query.tempdata.TempTableStore; /** * Defines the context that a command is processing in. For example, this defines @@ -103,12 +104,14 @@ private boolean validateXML; private BufferManager bufferManager; + + private TempTableStore globalTables; } private GlobalState globalState = new GlobalState(); private VariableContext variableContext = new VariableContext(); - private Object tempTableStore; + private TempTableStore tempTableStore; private LinkedList<String> recursionStack; /** @@ -180,10 +183,11 @@ globalState.processorID = object; } - public Object clone() { + public CommandContext clone() { CommandContext clone = new CommandContext(); clone.globalState = this.globalState; clone.variableContext = this.variableContext; + clone.tempTableStore = this.tempTableStore; if (this.recursionStack != null) { clone.recursionStack = new LinkedList<String>(this.recursionStack); } @@ -355,11 +359,11 @@ this.globalState.securityFunctionEvaluator = securityFunctionEvaluator; } - public Object getTempTableStore() { + public TempTableStore getTempTableStore() { return tempTableStore; } - public void setTempTableStore(Object tempTableStore) { + public void setTempTableStore(TempTableStore tempTableStore) { this.tempTableStore = tempTableStore; } @@ -454,5 +458,13 @@ public void setBufferManager(BufferManager bm) { globalState.bufferManager = bm; } + + public TempTableStore getGlobalTableStore() { + return globalState.globalTables; + } + + public void setGlobalTableStore(TempTableStore tempTableStore) { + globalState.globalTables = tempTableStore; + } } Deleted: trunk/engine/src/main/java/org/teiid/query/util/XMLFormatConstants.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/util/XMLFormatConstants.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/main/java/org/teiid/query/util/XMLFormatConstants.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -1,37 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.query.util; - - -/** - * @since 5.0 - */ -public interface XMLFormatConstants { - - /** XML results format: XML results displayed as a tree*/ - public static final String XML_TREE_FORMAT = "Tree"; //$NON-NLS-1$ - - /** XML results format: XML results displayed in compact form*/ - public static final String XML_COMPACT_FORMAT = "Compact"; //$NON-NLS-1$ - -} Modified: trunk/engine/src/test/java/org/teiid/common/buffer/TestSTree.java =================================================================== --- trunk/engine/src/test/java/org/teiid/common/buffer/TestSTree.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/common/buffer/TestSTree.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -28,7 +28,6 @@ import java.util.List; import org.junit.Test; -import org.teiid.common.buffer.BufferManager.TupleSourceType; import org.teiid.core.TeiidComponentException; import org.teiid.query.sql.symbol.ElementSymbol; @@ -42,7 +41,7 @@ ElementSymbol e2 = new ElementSymbol("y"); e2.setType(String.class); List elements = Arrays.asList(e1, e2); - STree map = bm.createSTree(elements, "1", TupleSourceType.PROCESSOR, 1); + STree map = bm.createSTree(elements, "1", 1); for (int i = 20000; i > 0; i--) { assertNull(map.insert(Arrays.asList(i, String.valueOf(i)), true)); Modified: trunk/engine/src/test/java/org/teiid/dqp/internal/process/SimpleQueryProcessorFactory.java =================================================================== --- trunk/engine/src/test/java/org/teiid/dqp/internal/process/SimpleQueryProcessorFactory.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/dqp/internal/process/SimpleQueryProcessorFactory.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -69,7 +69,7 @@ idGenerator, finder, AnalysisRecord.createNonRecordingRecord(), commandContext); - CommandContext copy = (CommandContext) commandContext.clone(); + CommandContext copy = commandContext.clone(); return new QueryProcessor(plan, copy, bufferMgr, dataMgr); } } Modified: trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java =================================================================== --- trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -73,7 +73,7 @@ setMustRegisterCommands(false); } - public TupleSource registerRequest(Object processorID, Command command, String modelName, String connectorBindingId, int nodeID) throws org.teiid.core.TeiidComponentException { + public TupleSource registerRequest(CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws org.teiid.core.TeiidComponentException { assertNotNull(connectorBindingId); Collection elements = ElementCollectorVisitor.getElements(command, true, true); @@ -84,7 +84,7 @@ fail("Query Contains a MultiSourceElement -- MultiSource expansion did not happen"); //$NON-NLS-1$ } } - return super.registerRequest(processorID, command, modelName, connectorBindingId, nodeID); + return super.registerRequest(context, command, modelName, connectorBindingId, nodeID); } } Copied: trunk/engine/src/test/java/org/teiid/query/optimizer/TestBatchedUpdatePlanner.java (from rev 2390, trunk/engine/src/test/java/org/teiid/query/optimizer/batch/TestBatchedUpdatePlanner.java) =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestBatchedUpdatePlanner.java (rev 0) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestBatchedUpdatePlanner.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -0,0 +1,230 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.optimizer; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +import org.teiid.api.exception.query.QueryMetadataException; +import org.teiid.api.exception.query.QueryPlannerException; +import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidProcessingException; +import org.teiid.query.analysis.AnalysisRecord; +import org.teiid.query.metadata.QueryMetadataInterface; +import org.teiid.query.optimizer.QueryOptimizer; +import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; +import org.teiid.query.optimizer.capabilities.DefaultCapabilitiesFinder; +import org.teiid.query.optimizer.capabilities.FakeCapabilitiesFinder; +import org.teiid.query.optimizer.capabilities.SourceCapabilities; +import org.teiid.query.parser.QueryParser; +import org.teiid.query.processor.BatchedUpdatePlan; +import org.teiid.query.processor.ProcessorPlan; +import org.teiid.query.processor.relational.AccessNode; +import org.teiid.query.processor.relational.BatchedUpdateNode; +import org.teiid.query.processor.relational.ProjectNode; +import org.teiid.query.processor.relational.RelationalNode; +import org.teiid.query.processor.relational.RelationalPlan; +import org.teiid.query.resolver.QueryResolver; +import org.teiid.query.rewriter.QueryRewriter; +import org.teiid.query.sql.lang.BatchedUpdateCommand; +import org.teiid.query.sql.lang.Command; +import org.teiid.query.unittest.FakeMetadataFactory; +import org.teiid.query.validator.Validator; +import org.teiid.query.validator.ValidatorReport; + +import junit.framework.TestCase; + + + +/** + * @since 4.2 + */ +public class TestBatchedUpdatePlanner extends TestCase { + + public TestBatchedUpdatePlanner(String name) { + super(name); + } + + public static List<Command> helpGetCommands(String[] sql, QueryMetadataInterface md) throws TeiidComponentException, TeiidProcessingException { + if(DEBUG) System.out.println("\n####################################\n" + sql); //$NON-NLS-1$ + List<Command> commands = new ArrayList<Command>(sql.length); + for (int i = 0; i < sql.length; i++) { + Command command = QueryParser.getQueryParser().parseCommand(sql[i]); + QueryResolver.resolveCommand(command, md); + ValidatorReport repo = Validator.validate(command, md); + Collection failures = new ArrayList(); + repo.collectInvalidObjects(failures); + if (failures.size() > 0){ + fail("Exception during validation (" + repo); //$NON-NLS-1$ + } + + command = QueryRewriter.rewrite(command, md, null); + commands.add(command); + } + return commands; + } + + private BatchedUpdateCommand helpGetCommand(String[] sql, QueryMetadataInterface md) throws TeiidComponentException, TeiidProcessingException { + BatchedUpdateCommand command = new BatchedUpdateCommand(helpGetCommands(sql, md)); + return command; + } + + private BatchedUpdatePlan helpPlanCommand(Command command, QueryMetadataInterface md, CapabilitiesFinder capFinder, boolean shouldSucceed) throws QueryPlannerException, QueryMetadataException, TeiidComponentException { + + // plan + ProcessorPlan plan = null; + AnalysisRecord analysisRecord = new AnalysisRecord(false, DEBUG); + if (shouldSucceed) { + try { + //do planning + plan = QueryOptimizer.optimizePlan(command, md, null, capFinder, analysisRecord, null); + + } finally { + if(DEBUG) { + System.out.println(analysisRecord.getDebugLog()); + } + } + return (BatchedUpdatePlan)plan; + } + Exception exception = null; + try { + //do planning + QueryOptimizer.optimizePlan(command, md, null, capFinder, analysisRecord, null); + + } catch (QueryPlannerException e) { + exception = e; + } catch (TeiidComponentException e) { + exception = e; + } finally { + if(DEBUG) { + System.out.println(analysisRecord.getDebugLog()); + } + } + assertNotNull("Expected exception but did not get one.", exception); //$NON-NLS-1$ + return null; + } + + public static CapabilitiesFinder getGenericFinder() { + return new DefaultCapabilitiesFinder(new FakeCapabilities(true)); + } + + private BatchedUpdatePlan helpPlan(String[] sql, QueryMetadataInterface md) throws TeiidComponentException, QueryMetadataException, TeiidProcessingException { + return helpPlan(sql, md, getGenericFinder(), true); + } + + private BatchedUpdatePlan helpPlan(String[] sql, QueryMetadataInterface md, CapabilitiesFinder capFinder, boolean shouldSucceed) throws TeiidComponentException, QueryMetadataException, TeiidProcessingException { + Command command = helpGetCommand(sql, md); + + if (capFinder == null){ + capFinder = getGenericFinder(); + } + + return helpPlanCommand(command, md, capFinder, shouldSucceed); + } + + private void helpAssertIsBatchedPlan(RelationalPlan plan, boolean isBatchedPlan) { + RelationalNode node = plan.getRootNode(); + if (node instanceof ProjectNode) { + node = node.getChildren()[0]; + } + if (isBatchedPlan) { + assertTrue("Plan should have been a batched", node instanceof BatchedUpdateNode); //$NON-NLS-1$ + } else { + assertTrue("Plan should not have been batched.", node instanceof AccessNode); //$NON-NLS-1$ + } + } + + private void helpTestPlanner(String[] sql, boolean[] expectedBatching) throws QueryMetadataException, TeiidComponentException, TeiidProcessingException { + BatchedUpdatePlan plan = helpPlan(sql, FakeMetadataFactory.example1Cached()); + List plans = plan.getUpdatePlans(); + assertEquals("Number of child plans did not match expected", expectedBatching.length, plans.size()); //$NON-NLS-1$ + for (int i = 0; i < expectedBatching.length; i++) { + helpAssertIsBatchedPlan((RelationalPlan)plans.get(i), expectedBatching[i]); + } + } + + private void helpTestPlanner(String[] sql, boolean[] expectedBatching, CapabilitiesFinder capFinder) throws QueryMetadataException, TeiidComponentException, TeiidProcessingException { + BatchedUpdatePlan plan = helpPlan(sql, FakeMetadataFactory.example1Cached(), capFinder, true); + List plans = plan.getUpdatePlans(); + assertEquals("Number of child plans did not match expected", expectedBatching.length, plans.size()); //$NON-NLS-1$ + for (int i = 0; i < expectedBatching.length; i++) { + helpAssertIsBatchedPlan((RelationalPlan)plans.get(i), expectedBatching[i]); + } + } + + public void testPlannerAllCommandsBatched() throws Exception { + String[] sql = {"INSERT INTO pm1.g1 (e1, e2, e3, e4) values ('string1', 1, {b'true'}, 1.0)", //$NON-NLS-1$ + "INSERT INTO pm1.g2 (e1, e2, e3, e4) values ('string1', 1, {b'true'}, 1.0)", //$NON-NLS-1$ + "DELETE FROM pm1.g1 WHERE e2 > 5000", //$NON-NLS-1$ + "UPDATE pm1.g1 set e2 = -1 WHERE e2 = 4999" //$NON-NLS-1$ + }; + boolean[] expectedBatching = {true}; + helpTestPlanner(sql, expectedBatching); + } + + public void testPlannerNoCommandsBatched() throws Exception { + String[] sql = {"INSERT INTO pm1.g1 (e1, e2, e3, e4) values ('string1', 1, {b'true'}, 1.0)", //$NON-NLS-1$ + "INSERT INTO pm1.g2 (e1, e2, e3, e4) values ('string1', 1, {b'true'}, 1.0)", //$NON-NLS-1$ + "DELETE FROM pm1.g1 WHERE e2 > 5000", //$NON-NLS-1$ + "UPDATE pm1.g1 set e2 = -1 WHERE e2 = 4999" //$NON-NLS-1$ + }; + FakeCapabilitiesFinder finder = new FakeCapabilitiesFinder(); + finder.addCapabilities("pm1", new FakeCapabilities(false)); //$NON-NLS-1$ + boolean[] expectedBatching = {false, false, false, false}; + helpTestPlanner(sql, expectedBatching, finder); + } + + public void testPlannerSomeCommandsBatched() throws Exception { + String[] sql = {"INSERT INTO pm1.g1 (e1, e2, e3, e4) values ('string1', 1, {b'true'}, 1.0)", //$NON-NLS-1$ + "INSERT INTO pm1.g2 (e1, e2, e3, e4) values ('string1', 1, {b'true'}, 1.0)", //$NON-NLS-1$ + "DELETE FROM pm2.g1 WHERE e2 > 5000", //$NON-NLS-1$ + "INSERT INTO pm2.g1 (e1, e2, e3, e4) values ('5000', 5000, {b'true'}, 5000.0)", //$NON-NLS-1$ + "UPDATE pm2.g1 set e2 = -1 WHERE e2 = 4999", //$NON-NLS-1$ + "DELETE FROM pm1.g2 WHERE e2 = 50" //$NON-NLS-1$ + }; + FakeCapabilitiesFinder finder = new FakeCapabilitiesFinder(); + finder.addCapabilities("pm1", new FakeCapabilities(false)); //$NON-NLS-1$ + finder.addCapabilities("pm2", new FakeCapabilities(true)); //$NON-NLS-1$ + boolean[] expectedBatching = {false, false, true, false}; + helpTestPlanner(sql, expectedBatching, finder); + } + + private static final class FakeCapabilities implements SourceCapabilities { + private boolean supportsBatching = false; + private FakeCapabilities(boolean supportsBatching) { + this.supportsBatching = supportsBatching; + } + public Scope getScope() {return null;} + public boolean supportsCapability(Capability capability) { + return !capability.equals(Capability.BATCHED_UPDATES) || supportsBatching; + } + public boolean supportsFunction(String functionName) {return false;} + // since 4.4 + public Object getSourceProperty(Capability propertyName) {return null;} + + } + + private static final boolean DEBUG = false; + +} Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -228,7 +228,7 @@ } // Collect atomic queries - ProcessorPlan plan = getPlan(command, md, capFinder, analysisRecord, mode != ComparisonMode.FAILED_PLANNING); + ProcessorPlan plan = getPlan(command, md, capFinder, analysisRecord, mode != ComparisonMode.FAILED_PLANNING, new CommandContext()); if (mode == ComparisonMode.CORRECTED_COMMAND_STRING) { checkAtomicQueries(expectedAtomic, plan, md, capFinder); @@ -279,49 +279,35 @@ assertEquals("Did not get expected atomic queries: ", expectedQueries, actualQueries); //$NON-NLS-1$ } - static ProcessorPlan getPlan(Command command, QueryMetadataInterface md, CapabilitiesFinder capFinder, AnalysisRecord analysisRecord, boolean shouldSucceed) { - // plan + public static ProcessorPlan getPlan(Command command, QueryMetadataInterface md, CapabilitiesFinder capFinder, AnalysisRecord analysisRecord, boolean shouldSucceed, CommandContext cc) { ProcessorPlan plan = null; if (analysisRecord == null) { analysisRecord = new AnalysisRecord(false, DEBUG); } - if (shouldSucceed) { - try { - //do planning - plan = QueryOptimizer.optimizePlan(command, md, null, capFinder, analysisRecord, new CommandContext()); - - } catch (Throwable e) { - throw new TeiidRuntimeException(e); - } finally { - if(DEBUG) { - System.out.println(analysisRecord.getDebugLog()); - } - } - } else { - Exception exception = null; - try { - //do planning - QueryOptimizer.optimizePlan(command, md, null, capFinder, analysisRecord, null); - - } catch (QueryPlannerException e) { - exception = e; - } catch (TeiidComponentException e) { - exception = e; - } catch (Throwable e) { - throw new TeiidRuntimeException(e); - } finally { - if(DEBUG) { - System.out.println(analysisRecord.getDebugLog()); - } - } + Exception exception = null; + try { + //do planning + plan = QueryOptimizer.optimizePlan(command, md, null, capFinder, analysisRecord, cc); + } catch (QueryPlannerException e) { + exception = e; + } catch (TeiidComponentException e) { + exception = e; + } catch (Throwable e) { + throw new TeiidRuntimeException(e); + } finally { + if(DEBUG) { + System.out.println(analysisRecord.getDebugLog()); + } + } + if (!shouldSucceed) { assertNotNull("Expected exception but did not get one.", exception); //$NON-NLS-1$ return null; + } + if (plan == null) { + throw new TeiidRuntimeException(exception); } - - assertNotNull("Output elements are null", plan.getOutputElements()); //$NON-NLS-1$ - + assertNotNull("Output elements are null", plan.getOutputElements()); //$NON-NLS-1$ if(DEBUG) System.out.println("\n" + plan); //$NON-NLS-1$ - return plan; } Modified: trunk/engine/src/test/java/org/teiid/query/processor/FakeDataManager.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/FakeDataManager.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/FakeDataManager.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -108,10 +108,10 @@ // does nothing? } - public TupleSource registerRequest(Object processorID, Command command, String modelName, String connectorBindingId, int nodeID) + public TupleSource registerRequest(CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException { - LogManager.logTrace(LOG_CONTEXT, new Object[]{"Register Request:", command, ",processorID:", processorID, ",model name:", modelName,",TupleSourceID nodeID:",new Integer(nodeID)}); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ + LogManager.logTrace(LOG_CONTEXT, new Object[]{"Register Request:", command, ",processorID:", context.getProcessorID(), ",model name:", modelName,",TupleSourceID nodeID:",new Integer(nodeID)}); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ if (this.recordingCommands) { if (! (command instanceof BatchedUpdateCommand) ) { Modified: trunk/engine/src/test/java/org/teiid/query/processor/HardcodedDataManager.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/HardcodedDataManager.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/HardcodedDataManager.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -110,10 +110,10 @@ } /** - * @see org.teiid.query.processor.ProcessorDataManager#registerRequest(java.lang.Object, org.teiid.query.sql.lang.Command, java.lang.String, String, int) + * @see org.teiid.query.processor.ProcessorDataManager#registerRequest(CommandContext, org.teiid.query.sql.lang.Command, java.lang.String, String, int) * @since 4.2 */ - public TupleSource registerRequest(Object processorID, + public TupleSource registerRequest(CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException { Copied: trunk/engine/src/test/java/org/teiid/query/processor/TestBatchedUpdatePlan.java (from rev 2390, trunk/engine/src/test/java/org/teiid/query/processor/batch/TestBatchedUpdatePlan.java) =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestBatchedUpdatePlan.java (rev 0) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestBatchedUpdatePlan.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -0,0 +1,85 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.processor; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.teiid.common.buffer.TupleBatch; +import org.teiid.query.processor.BatchedUpdatePlan; + +import junit.framework.TestCase; + + + +/** + * @since 4.2 + */ +public class TestBatchedUpdatePlan extends TestCase { + + public TestBatchedUpdatePlan(String name) { + super(name); + } + + private void helpTestNextBatch(int[] commandsPerPlan) throws Exception { + List plans = new ArrayList(commandsPerPlan.length); + int totalCommands = 0; + for (int i = 0; i < commandsPerPlan.length; i++) { + totalCommands += commandsPerPlan[i]; + plans.add(new FakeProcessorPlan(commandsPerPlan[i])); + } + BatchedUpdatePlan plan = new BatchedUpdatePlan(plans, totalCommands, null); + TupleBatch batch = plan.nextBatch(); + assertEquals(totalCommands, batch.getRowCount()); + for (int i = 1; i <= totalCommands; i++) { + assertEquals(new Integer(1), batch.getTuple(i).get(0)); + } + } + + public void testOpen() throws Exception { + FakeProcessorPlan[] plans = new FakeProcessorPlan[4]; + for (int i = 0; i < plans.length; i++) { + plans[i] = new FakeProcessorPlan(1); + } + BatchedUpdatePlan plan = new BatchedUpdatePlan(Arrays.asList(plans), plans.length, null); + plan.open(); + // First plan may or may not be opened, but all subsequent plans should not be opened. + for (int i = 1; i < plans.length; i++) { + assertFalse(plans[i].isOpened()); + } + } + + public void testNextBatch1() throws Exception { + helpTestNextBatch(new int[] {1, 5, 2, 1, 10, 1, 1}); + } + + public void testNextBatch2() throws Exception { + helpTestNextBatch(new int[] {5, 4, 10, 7, 22, 9, 12, 8, 11}); + } + + public void testNextBatch3() throws Exception { + helpTestNextBatch(new int[] {1, 1, 1, 1}); + } + +} Modified: trunk/engine/src/test/java/org/teiid/query/processor/TestProcedureRelational.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestProcedureRelational.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestProcedureRelational.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -50,6 +50,7 @@ import org.teiid.query.unittest.FakeMetadataFactory; import org.teiid.query.unittest.FakeMetadataObject; import org.teiid.query.unittest.FakeMetadataStore; +import org.teiid.query.util.CommandContext; public class TestProcedureRelational { @@ -723,7 +724,7 @@ // Run query HardcodedDataManager dataManager = new HardcodedDataManager() { @Override - public TupleSource registerRequest(Object processorID, + public TupleSource registerRequest(CommandContext context, Command command, String modelName, String connectorBindingId, int nodeID) throws TeiidComponentException { @@ -736,7 +737,7 @@ Arrays.asList(value+2), Arrays.asList(value+5) }); } - return super.registerRequest(processorID, command, modelName, + return super.registerRequest(context, command, modelName, connectorBindingId, nodeID); } }; Modified: trunk/engine/src/test/java/org/teiid/query/processor/TestTempTables.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestTempTables.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestTempTables.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -51,7 +51,7 @@ } @Before public void setUp() { - TempTableStore tempStore = new TempTableStore(BufferManagerFactory.getStandaloneBufferManager(), "1", null); //$NON-NLS-1$ + TempTableStore tempStore = new TempTableStore(BufferManagerFactory.getStandaloneBufferManager(), "1"); //$NON-NLS-1$ metadata = new TempMetadataAdapter(FakeMetadataFactory.example1Cached(), tempStore.getMetadataStore()); FakeDataManager fdm = new FakeDataManager(); TestProcessor.sampleData1(fdm); Modified: trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -37,7 +37,7 @@ import org.teiid.common.buffer.TupleSource; import org.teiid.core.TeiidComponentException; import org.teiid.query.metadata.QueryMetadataInterface; -import org.teiid.query.optimizer.batch.TestBatchedUpdatePlanner; +import org.teiid.query.optimizer.TestBatchedUpdatePlanner; import org.teiid.query.processor.ProcessorDataManager; import org.teiid.query.sql.lang.BatchedUpdateCommand; import org.teiid.query.sql.lang.Command; @@ -211,8 +211,8 @@ this.numExecutedCommands = numExecutedCommands; } public Object lookupCodeValue(CommandContext context,String codeTableName,String returnElementName,String keyElementName,Object keyValue) throws BlockedException,TeiidComponentException {return null;} - public TupleSource registerRequest(Object processorID,Command command,String modelName,String connectorBindingId, int nodeID) throws TeiidComponentException { - assertEquals("myProcessorID", processorID); //$NON-NLS-1$ + public TupleSource registerRequest(CommandContext context,Command command,String modelName,String connectorBindingId, int nodeID) throws TeiidComponentException { + assertEquals("myProcessorID", context.getProcessorID()); //$NON-NLS-1$ assertEquals("myModelName", modelName); //$NON-NLS-1$ assertEquals(1, nodeID); commands.add(command.toString()); Modified: trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -132,7 +132,7 @@ this.expectedBatchSize = expectedBatchSize; } public Object lookupCodeValue(CommandContext context,String codeTableName,String returnElementName,String keyElementName,Object keyValue) throws BlockedException,TeiidComponentException {return null;} - public TupleSource registerRequest(Object processorID,Command command,String modelName,String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException { + public TupleSource registerRequest(CommandContext context,Command command,String modelName,String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException { callCount++; int batchSize = 1; Modified: trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -548,7 +548,7 @@ List normDocE3 = FakeMetadataFactory.createElements(normDoc3, new String[] { "Catalogs", "Catalogs.Catalog", "Catalogs.Catalog.items", "Catalogs.Catalog.items.item", "Catalogs.Catalog.items.item.@ItemID", "Catalogs.Catalog.items.item.Name", "Catalogs.Catalog.items.item.Quantity", "Catalogs.Catalog.items.DiscontinuedItem", "Catalogs.Catalog.items.DiscontinuedItem.@ItemID", "Catalogs.Catalog.items.DiscontinuedItem.Name", "Catalogs.Catalog.items.DiscontinuedItem.Quantity", "Catalogs.Catalog.items.StatusUnknown", "Catalogs.Catalog.items.StatusUnknown.@ItemID", "Catalogs.Catalog.items.StatusUnknown.Name", "Catalogs.Catalog.items.StatusUnknown.Quantity", "Catalogs.Catalog.items.Shouldn't see", "Catalogs.Catalog.items.Shouldn't see 2" }, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ //$NON-NLS-6$ //$NON-NLS-7$ //$NON-NLS-8$ //$NON-NLS-9$ //$NON-NLS-10$ //$NON-NLS-11$ //$NON-NLS-12$ //$NON-NLS-13$ //$NON-NLS-14$ //$NON-NLS-15$ //$NON-NLS-16$ //$NON-N! LS-17$ new String[] { DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.INTEGER, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.INTEGER, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.INTEGER, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING }); - QueryNode vspqn1 = new QueryNode("vsp1", "CREATE VIRTUAL PROCEDURE BEGIN SELECT * FROM xmltest.doc1; END"); //$NON-NLS-1$ //$NON-NLS-2$ + QueryNode vspqn1 = new QueryNode("vsp1", "CREATE VIRTUAL PROCEDURE BEGIN insert into #temp select * from stock.items; SELECT * FROM xmltest.doc1 where Item.Quantity < (select avg(itemquantity) from #temp); END"); //$NON-NLS-1$ //$NON-NLS-2$ FakeMetadataObject vsprs1 = FakeMetadataFactory.createResultSet("pm1.vsprs1", xmltest, new String[] { "xml" }, new String[] { DataTypeManager.DefaultDataTypes.XML }); //$NON-NLS-1$ //$NON-NLS-2$ FakeMetadataObject vspp1 = FakeMetadataFactory.createParameter("ret", 1, ParameterInfo.RESULT_SET, DataTypeManager.DefaultDataTypes.XML, vsprs1); //$NON-NLS-1$ FakeMetadataObject vsp1 = FakeMetadataFactory.createVirtualProcedure("xmltest.vsp1", xmltest, Arrays.asList(new FakeMetadataObject[] { vspp1 }), vspqn1); //$NON-NLS-1$ @@ -11847,7 +11847,7 @@ } /** - * Also tests the logic of the {@link XMLPostProcessor} + * Ensures that temp tables are still visible when processing criteria */ @Test public void testProcedureAndXML() throws Exception { FakeMetadataFacade metadata = exampleMetadataCached(); @@ -11861,10 +11861,6 @@ " <Name>Lamp</Name>\r\n" + //$NON-NLS-1$ " <Quantity>5</Quantity>\r\n" + //$NON-NLS-1$ " </Item>\r\n" + //$NON-NLS-1$ - " <Item ItemID=\"002\">\r\n" + //$NON-NLS-1$ - " <Name>Screwdriver</Name>\r\n" + //$NON-NLS-1$ - " <Quantity>100</Quantity>\r\n" + //$NON-NLS-1$ - " </Item>\r\n" + //$NON-NLS-1$ " <Item ItemID=\"003\">\r\n" + //$NON-NLS-1$ " <Name>Goat</Name>\r\n" + //$NON-NLS-1$ " <Quantity>4</Quantity>\r\n" + //$NON-NLS-1$ @@ -11872,6 +11868,7 @@ " </Items>\r\n" + //$NON-NLS-1$ " </Catalog>\r\n" + //$NON-NLS-1$ "</Catalogs>\r\n\r\n"; //$NON-NLS-1$ + helpTestProcess("call xmltest.vsp1()", expectedDoc, metadata, dataMgr); //$NON-NLS-1$ } /** Modified: trunk/engine/src/test/java/org/teiid/query/unittest/FakeMetadataFactory.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/unittest/FakeMetadataFactory.java 2010-07-30 20:39:58 UTC (rev 2394) +++ trunk/engine/src/test/java/org/teiid/query/unittest/FakeMetadataFactory.java 2010-07-31 22:31:31 UTC (rev 2395) @@ -2127,6 +2127,14 @@ virtGroup1.putProperty(FakeMetadataObject.Props.MAT_GROUP, physGroup1); virtGroup1.putProperty(FakeMetadataObject.Props.MAT_STAGE, physGroupStage1); + //add one virtual group that uses the materialized group in transformation with NOCACHE option + QueryNode vTrans2 = new QueryNode("MatView.VGroup2", "/* cache */ SELECT x FROM matsrc"); //$NON-NLS-1$ //$NON-NLS-2$ + FakeMetadataObject vGroup2 = createVirtualGroup("MatView.VGroup2", virtModel, vTrans2); //$NON-NLS-1$ + List vElements2 = createElements(vGroup2, + new String[] { "x" }, //$NON-NLS-1$ + new String[] { DataTypeManager.DefaultDataTypes.STRING}); + + FakeMetadataStore store = new FakeMetadataStore(); store.addObject(virtModel); store.addObject(physModel); @@ -2147,6 +2155,8 @@ store.addObjects(vElements); store.addObject(virtGroup1); store.addObjects(virtElements1); + store.addObject(vGroup2); + store.addObjects(vElements2); return new FakeMetadataFacade(store); }

14 years, 4 months

1
0
0 / 0

teiid SVN: r2394 - trunk/console/src/main/java/org/teiid/rhq/plugin.

by teiid-commits＠lists.jboss.org

Author: rareddy Date: 2010-07-30 16:39:58 -0400 (Fri, 30 Jul 2010) New Revision: 2394 Modified: trunk/console/src/main/java/org/teiid/rhq/plugin/Facet.java Log: TEIID-1178: Since we do not support any versioning in the Teiid yet, package names are not supported currently. Modified: trunk/console/src/main/java/org/teiid/rhq/plugin/Facet.java =================================================================== --- trunk/console/src/main/java/org/teiid/rhq/plugin/Facet.java 2010-07-30 20:30:08 UTC (rev 2393) +++ trunk/console/src/main/java/org/teiid/rhq/plugin/Facet.java 2010-07-30 20:39:58 UTC (rev 2394) @@ -23,6 +23,7 @@ import java.io.File; import java.io.InputStream; +import java.util.Collections; import java.util.HashMap; import java.util.HashSet; import java.util.List; @@ -418,43 +419,8 @@ @Override public Set<ResourcePackageDetails> discoverDeployedPackages(PackageType arg0) { - - File deploymentFile = null; - - if (this.deploymentName != null) { - deploymentFile = new File(deploymentName.substring(7)); - } - - if (!deploymentFile.exists()) - throw new IllegalStateException("Deployment file '" - + deploymentFile + "' for " + this.getComponentType() - + " does not exist."); - - String fileName = deploymentFile.getName(); - org.rhq.core.pluginapi.content.version.PackageVersions packageVersions = loadPackageVersions(); - String version = packageVersions.getVersion(fileName); - if (version == null) { - // This is either the first time we've discovered this VDB, or - // someone purged the PC's data dir. - version = "1.0"; - packageVersions.putVersion(fileName, version); - packageVersions.saveToDisk(); - } - - // Package name is the deployment's file name (e.g. foo.ear). - PackageDetailsKey key = new PackageDetailsKey(fileName, version, - PKG_TYPE_VDB, ARCHITECTURE); - ResourcePackageDetails packageDetails = new ResourcePackageDetails(key); - packageDetails.setFileName(fileName); - packageDetails.setLocation(deploymentFile.getPath()); - if (!deploymentFile.isDirectory()) - packageDetails.setFileSize(deploymentFile.length()); - packageDetails.setFileCreatedDate(null); // TODO: get created date via - // SIGAR - Set<ResourcePackageDetails> packages = new HashSet<ResourcePackageDetails>(); - packages.add(packageDetails); - - return packages; + // Teiid does not support the versions yet. + return Collections.EMPTY_SET; } @Override

14 years, 4 months

1
0
0 / 0

teiid SVN: r2393 - in trunk: engine/src/main/java/org/teiid/query/resolver/command and 1 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2010-07-30 16:30:08 -0400 (Fri, 30 Jul 2010) New Revision: 2393 Modified: trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml trunk/engine/src/main/java/org/teiid/query/resolver/command/InsertResolver.java trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java Log: TEIID-1177 refining the resolving logic to omit the columns for defining a temp table with a query expression Modified: trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml =================================================================== --- trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml 2010-07-30 20:06:10 UTC (rev 2392) +++ trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml 2010-07-30 20:30:08 UTC (rev 2393) @@ -647,13 +647,21 @@ <para>Creation syntax: </para> <listitem> - <para>CREATE LOCAL TEMPORARY TABLE x (column type, ... [PRIMARY KEY (column, ...)]) + <para>Explicit: CREATE LOCAL TEMPORARY TABLE x (column type, ... [PRIMARY KEY (column, ...)]) </para> </listitem> <listitem> - <para>INSERT INTO #x (column, ...) VALUES (value, ...) + <para>Implicit: INSERT INTO #x (column, ...) VALUES (value, ...) </para> + <para>If #x doen't exist, it will be defined using the given column names and types from the value expressions. + </para> </listitem> + <listitem> + <para>Implicit: INSERT INTO #x [(column, ...)] select c1, c2 from t + </para> + <para>If #x doesn't exist, it will be defined using the target column names (in not supplied, the column names will match the derived column names from the query), and the types from the query derived columns. + </para> + </listitem> </itemizedlist> <itemizedlist> <para>Drop syntax: Modified: trunk/engine/src/main/java/org/teiid/query/resolver/command/InsertResolver.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/resolver/command/InsertResolver.java 2010-07-30 20:06:10 UTC (rev 2392) +++ trunk/engine/src/main/java/org/teiid/query/resolver/command/InsertResolver.java 2010-07-30 20:30:08 UTC (rev 2393) @@ -56,6 +56,7 @@ import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.Reference; +import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.util.ErrorMessageKeys; @@ -88,6 +89,14 @@ Set<GroupSymbol> groups = new HashSet<GroupSymbol>(); groups.add(insert.getGroup()); + // resolve any functions in the values + List values = insert.getValues(); + boolean usingQuery = insert.getQueryExpression() != null; + + if (usingQuery) { + values = insert.getQueryExpression().getProjectedSymbols(); + } + if (insert.getVariables().isEmpty()) { if (insert.getGroup().isResolved()) { List variables = ResolverUtil.resolveElementsInGroup(insert.getGroup(), metadata); @@ -95,15 +104,22 @@ insert.addVariable((ElementSymbol)((ElementSymbol)i.next()).clone()); } } else { - for (int i = 0; i < insert.getValues().size(); i++) { - insert.addVariable(new ElementSymbol("expr" + i)); //$NON-NLS-1$ + for (int i = 0; i < values.size(); i++) { + if (usingQuery) { + SingleElementSymbol ses = (SingleElementSymbol)values.get(i); + ElementSymbol es = new ElementSymbol(ses.getShortName()); + es.setType(ses.getType()); + insert.addVariable(es); + } else { + insert.addVariable(new ElementSymbol("expr" + i)); //$NON-NLS-1$ + } } } } else if (insert.getGroup().isResolved()) { resolveVariables(metadata, insert, groups); } - resolveTypes(insert, metadata); + resolveTypes(insert, metadata, values, usingQuery); if (!insert.getGroup().isResolved()) { //define the implicit temp group if(insert.getQueryExpression() != null) { @@ -114,7 +130,7 @@ resolveVariables(metadata, insert, groups); //ensure that the types match - resolveTypes(insert, metadata); + resolveTypes(insert, metadata, values, usingQuery); } if (insert.getQueryExpression() != null && metadata.isVirtualGroup(insert.getGroup().getMetadataID())) { @@ -150,24 +166,17 @@ /** * @param insert + * @param values + * @param usingQuery * @throws QueryResolverException */ - public void resolveTypes(Insert insert, TempMetadataAdapter metadata) throws QueryResolverException { + public void resolveTypes(Insert insert, TempMetadataAdapter metadata, List values, boolean usingQuery) throws QueryResolverException { - boolean usingQuery = insert.getQueryExpression() != null; - - // resolve any functions in the values - List values = insert.getValues(); - - if (usingQuery) { - values = insert.getQueryExpression().getProjectedSymbols(); - } - List newValues = new ArrayList(values.size()); // check that # of variables == # of values if(values.size() != insert.getVariables().size()) { - throw new QueryResolverException(ErrorMessageKeys.RESOLVER_0010, QueryPlugin.Util.getString(ErrorMessageKeys.RESOLVER_0010, new Object[] {new Integer(insert.getVariables().size()), new Integer(insert.getValues().size())})); + throw new QueryResolverException(ErrorMessageKeys.RESOLVER_0010, QueryPlugin.Util.getString(ErrorMessageKeys.RESOLVER_0010, insert.getVariables().size(), values.size())); } Iterator valueIter = values.iterator(); Modified: trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java 2010-07-30 20:06:10 UTC (rev 2392) +++ trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java 2010-07-30 20:30:08 UTC (rev 2393) @@ -2706,6 +2706,17 @@ helpResolveException(sql, "Cannot convert insert query expression projected symbol '3' of type java.lang.Integer to insert column 'pm1.g1.e3' of type java.lang.Boolean"); //$NON-NLS-1$ } + + @Test public void testInsertWithQueryImplicitWithoutColumns() { + String sql = "Insert into #X select 1 as x, 2 as y, 3 as z"; //$NON-NLS-1$ + helpResolve(sql); //$NON-NLS-1$ + } + + @Test public void testInsertWithQueryImplicitWithoutColumns1() { + String sql = "Insert into #X select 1 as x, 2 as y, 3 as y"; //$NON-NLS-1$ + + helpResolveException(sql, "Cannot create group '#X' with multiple columns named 'y'"); //$NON-NLS-1$ + } @Test public void testInsertWithoutColumnsPasses() { String sql = "Insert into pm1.g1 values (1, 2, true, 4)"; //$NON-NLS-1$

14 years, 4 months

1
0
0 / 0

teiid SVN: r2392 - in trunk: common-core/src/main/java/org/teiid/core and 7 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2010-07-30 16:06:10 -0400 (Fri, 30 Jul 2010) New Revision: 2392 Added: trunk/engine/src/main/java/org/teiid/core/ trunk/engine/src/main/java/org/teiid/core/id/ trunk/engine/src/test/java/org/teiid/core/ trunk/engine/src/test/java/org/teiid/core/id/ trunk/engine/src/test/resources/UUIDs.txt Removed: trunk/common-core/src/main/java/org/teiid/core/id/ trunk/common-core/src/test/java/org/teiid/core/id/ trunk/common-core/src/test/resources/UUIDs.txt Modified: trunk/api/src/main/java/org/teiid/metadata/MetadataFactory.java Log: removing id framework from core, it is not needed there Modified: trunk/api/src/main/java/org/teiid/metadata/MetadataFactory.java =================================================================== --- trunk/api/src/main/java/org/teiid/metadata/MetadataFactory.java 2010-07-29 15:18:33 UTC (rev 2391) +++ trunk/api/src/main/java/org/teiid/metadata/MetadataFactory.java 2010-07-30 20:06:10 UTC (rev 2392) @@ -27,9 +27,9 @@ import java.util.List; import java.util.Map; import java.util.Properties; +import java.util.UUID; import org.teiid.connector.DataPlugin; -import org.teiid.core.id.UUIDFactory; import org.teiid.translator.TranslatorException; import org.teiid.translator.TypeFacility; @@ -43,7 +43,6 @@ public class MetadataFactory { private Schema schema; - private UUIDFactory factory = new UUIDFactory(); private Map<String, Datatype> dataTypes; private Properties importProperties; private MetadataStore store = new MetadataStore(); @@ -66,7 +65,7 @@ } private void setUUID(AbstractMetadataRecord record) { - record.setUUID(factory.create().toString()); + record.setUUID("mmuuid:" +UUID.randomUUID()); //$NON-NLS-1$ } /** Deleted: trunk/common-core/src/test/resources/UUIDs.txt =================================================================== --- trunk/common-core/src/test/resources/UUIDs.txt 2010-07-29 15:18:33 UTC (rev 2391) +++ trunk/common-core/src/test/resources/UUIDs.txt 2010-07-30 20:06:10 UTC (rev 2392) @@ -1,39 +0,0 @@ -# Some UUIDs from old license files -mmuuid:d17fd882-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d17fd883-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d17fd884-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d17fd885-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d17fd886-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba40-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba41-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba42-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba43-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba44-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba45-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba46-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba47-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba48-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba49-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba4a-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba4b-804f-1ec7-bc2e-8e296fe7f539 -mmuuid:d264ba4c-804f-1ec7-bc2e-8e296fe7f539 - -# Some other UUIDs with different delims -mmuuid/d37fd882-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d37fd883-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d37fd884-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d37fd885-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d37fd886-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba40-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba41-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba42-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba43-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba44-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba45-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba46-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba47-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba48-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba49-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba4a-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba4b-804f-1ec7-bc2e-8e296fe7f539 -mmuuid/d364ba4c-804f-1ec7-bc2e-8e296fe7f539 Copied: trunk/engine/src/main/java/org/teiid/core/id (from rev 2390, trunk/common-core/src/main/java/org/teiid/core/id) Copied: trunk/engine/src/test/java/org/teiid/core/id (from rev 2390, trunk/common-core/src/test/java/org/teiid/core/id) Copied: trunk/engine/src/test/resources/UUIDs.txt (from rev 2390, trunk/common-core/src/test/resources/UUIDs.txt) =================================================================== --- trunk/engine/src/test/resources/UUIDs.txt (rev 0) +++ trunk/engine/src/test/resources/UUIDs.txt 2010-07-30 20:06:10 UTC (rev 2392) @@ -0,0 +1,39 @@ +# Some UUIDs from old license files +mmuuid:d17fd882-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d17fd883-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d17fd884-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d17fd885-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d17fd886-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba40-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba41-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba42-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba43-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba44-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba45-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba46-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba47-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba48-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba49-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba4a-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba4b-804f-1ec7-bc2e-8e296fe7f539 +mmuuid:d264ba4c-804f-1ec7-bc2e-8e296fe7f539 + +# Some other UUIDs with different delims +mmuuid/d37fd882-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d37fd883-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d37fd884-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d37fd885-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d37fd886-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba40-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba41-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba42-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba43-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba44-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba45-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba46-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba47-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba48-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba49-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba4a-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba4b-804f-1ec7-bc2e-8e296fe7f539 +mmuuid/d364ba4c-804f-1ec7-bc2e-8e296fe7f539

14 years, 4 months

1
0
0 / 0

teiid SVN: r2391 - in trunk: api/src/main/java/org/teiid/language/visitor and 27 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2010-07-29 11:18:33 -0400 (Thu, 29 Jul 2010) New Revision: 2391 Added: trunk/api/src/main/java/org/teiid/language/IteratorValueSource.java trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java Removed: trunk/engine/src/main/java/org/teiid/query/optimizer/batch/ trunk/engine/src/test/java/org/teiid/cache/ Modified: trunk/api/src/main/java/org/teiid/language/visitor/AbstractLanguageVisitor.java trunk/api/src/main/java/org/teiid/language/visitor/CollectorVisitor.java trunk/api/src/main/java/org/teiid/language/visitor/LanguageObjectVisitor.java trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java trunk/build/kits/jboss-container/teiid-releasenotes.html trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCExecutionFactory.java trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCUpdateExecution.java trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/TranslatedCommand.java trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/modeshape/ModeShapeExecutionFactory.java trunk/connectors/translator-jdbc/src/test/java/org/teiid/translator/jdbc/TestJDBCUpdateExecution.java trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml trunk/documentation/reference/src/main/docbook/en-US/content/translators.xml trunk/engine/src/main/java/org/teiid/common/buffer/AbstractTupleSource.java trunk/engine/src/main/java/org/teiid/common/buffer/STree.java trunk/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java trunk/engine/src/main/java/org/teiid/common/buffer/TupleBuffer.java trunk/engine/src/main/java/org/teiid/common/buffer/TupleSource.java trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierTupleSource.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/PreparedStatementRequest.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java trunk/engine/src/main/java/org/teiid/query/function/metadata/FunctionMethod.java trunk/engine/src/main/java/org/teiid/query/function/source/SystemSource.java trunk/engine/src/main/java/org/teiid/query/metadata/TempCapabilitiesFinder.java trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java trunk/engine/src/main/java/org/teiid/query/processor/BatchIterator.java trunk/engine/src/main/java/org/teiid/query/processor/CollectionTupleSource.java trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentValueSource.java trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/RelationalPlan.java trunk/engine/src/main/java/org/teiid/query/processor/relational/SortNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java trunk/engine/src/main/java/org/teiid/query/processor/relational/SortingFilter.java trunk/engine/src/main/java/org/teiid/query/processor/relational/SourceState.java trunk/engine/src/main/java/org/teiid/query/sql/lang/Insert.java trunk/engine/src/main/java/org/teiid/query/sql/lang/Query.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java trunk/engine/src/test/java/org/teiid/query/processor/FakeTupleSource.java trunk/engine/src/test/java/org/teiid/query/processor/TestCollectionTupleSource.java trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java trunk/engine/src/test/java/org/teiid/query/processor/relational/TestGroupingNode.java trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java trunk/engine/src/test/java/org/teiid/query/processor/relational/TestSortNode.java Log: TEIID-892 introduced a way better control large inserts. cleaned up the tuplesource interface. TEIID-1167 changed temp tables to support the iterator based inserts since they do not support bulk delete/update logic. Added: trunk/api/src/main/java/org/teiid/language/IteratorValueSource.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/IteratorValueSource.java (rev 0) +++ trunk/api/src/main/java/org/teiid/language/IteratorValueSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -0,0 +1,58 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.language; + +import java.util.Iterator; +import java.util.List; + +import org.teiid.language.visitor.LanguageObjectVisitor; + +public class IteratorValueSource<T extends List<?>> extends BaseLanguageObject implements InsertValueSource { + + private Iterator<T> iter; + private int columnCount; + + public IteratorValueSource(Iterator<T> iter, int columnCount) { + this.iter = iter; + this.columnCount = columnCount; + } + + /** + * A memory safe iterator of the insert values. Only 1 iterator is associated + * with the value source. Once it is consumed there are no more values. + * @return + */ + public Iterator<T> getIterator() { + return iter; + } + + public int getColumnCount() { + return columnCount; + } + + @Override + public void acceptVisitor(LanguageObjectVisitor visitor) { + visitor.visit(this); + } + +} Property changes on: trunk/api/src/main/java/org/teiid/language/IteratorValueSource.java ___________________________________________________________________ Name: svn:mime-type + text/plain Modified: trunk/api/src/main/java/org/teiid/language/visitor/AbstractLanguageVisitor.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/visitor/AbstractLanguageVisitor.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/api/src/main/java/org/teiid/language/visitor/AbstractLanguageVisitor.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -41,6 +41,7 @@ import org.teiid.language.In; import org.teiid.language.Insert; import org.teiid.language.IsNull; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Join; import org.teiid.language.LanguageObject; import org.teiid.language.Like; @@ -146,4 +147,5 @@ public void visit(SetQuery obj) {} public void visit(SetClause obj) {} public void visit(SearchedWhenClause obj) {} + public void visit(IteratorValueSource obj) {} } Modified: trunk/api/src/main/java/org/teiid/language/visitor/CollectorVisitor.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/visitor/CollectorVisitor.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/api/src/main/java/org/teiid/language/visitor/CollectorVisitor.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -44,6 +44,7 @@ import org.teiid.language.In; import org.teiid.language.Insert; import org.teiid.language.IsNull; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Join; import org.teiid.language.LanguageObject; import org.teiid.language.Like; @@ -226,6 +227,11 @@ public void visit(SearchedWhenClause obj) { checkInstance(obj); } + + @Override + public void visit(IteratorValueSource obj) { + checkInstance(obj); + } /** * This is a utility method to instantiate and run the visitor in conjunction Modified: trunk/api/src/main/java/org/teiid/language/visitor/LanguageObjectVisitor.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/visitor/LanguageObjectVisitor.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/api/src/main/java/org/teiid/language/visitor/LanguageObjectVisitor.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -61,4 +61,5 @@ public void visit(SetQuery obj); public void visit(SetClause obj); public void visit(SearchedWhenClause obj); + public void visit(IteratorValueSource obj); } Modified: trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -50,6 +50,7 @@ import org.teiid.language.In; import org.teiid.language.Insert; import org.teiid.language.IsNull; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Join; import org.teiid.language.LanguageObject; import org.teiid.language.Like; @@ -494,6 +495,18 @@ append(obj.getValues()); buffer.append(Tokens.RPAREN); } + + @Override + public void visit(IteratorValueSource obj) { + buffer.append(VALUES).append(Tokens.SPACE).append(Tokens.LPAREN); + for (int i = 0; i < obj.getColumnCount(); i++) { + if (i > 0) { + buffer.append(", "); //$NON-NLS-1$ + } + buffer.append("?"); //$NON-NLS-1$ + } + buffer.append(Tokens.RPAREN); + } public void visit(IsNull obj) { append(obj.getExpression()); Modified: trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java =================================================================== --- trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -23,6 +23,7 @@ package org.teiid.translator; import java.util.Collection; +import java.util.Iterator; import java.util.List; import javax.resource.ResourceException; @@ -36,6 +37,7 @@ import org.teiid.language.Command; import org.teiid.language.LanguageFactory; import org.teiid.language.QueryExpression; +import org.teiid.language.Select; import org.teiid.language.SetQuery; import org.teiid.logging.LogConstants; import org.teiid.logging.LogManager; @@ -730,12 +732,22 @@ /** * Support indicates that the connector can accept INSERTs with - * values specified by an {@link SetQuery}. + * values specified by a {@link SetQuery} or {@link Select} * @since 6.1 */ public boolean supportsInsertWithQueryExpression() { return false; } + + /** + * Support indicates that the connector can accept INSERTs + * with values specified by an {@link Iterator} + * @since 7.1 + * @return + */ + public boolean supportsInsertWithIterator() { + return false; + } public static <T> T getInstance(Class<T> expectedType, String className, Collection<?> ctorObjs, Class<? extends T> defaultClass) throws TranslatorException { try { Modified: trunk/build/kits/jboss-container/teiid-releasenotes.html =================================================================== --- trunk/build/kits/jboss-container/teiid-releasenotes.html 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/build/kits/jboss-container/teiid-releasenotes.html 2010-07-29 15:18:33 UTC (rev 2391) @@ -10,7 +10,7 @@ <P><A HREF="http://www.teiid.org/"><IMG SRC="https://www.jboss.org/dms/teiid/images/teiid_banner.png" NAME="graphics1" ALT="Teiid" ALIGN=BOTTOM WIDTH=800></A> <H1>Teiid ${project.version} Release Notes</H1> -<P>Teiid ${project.version} adds ODBC, SQL, and caching features. +<P>Teiid ${project.version} adds ODBC, SQL, performance, and caching features. <H2>Overview</H2> <UL> @@ -38,7 +38,11 @@ <li>Added PRIMARY KEY and the associated index support to temp tables. </ul> <LI><B>Parallel Source Queries</B> - reestablished parallel execution of source queries within a query plan along with a prioritized work system to help prevent resource contention. + <LI><B>Improved Insert Support</B> - Cross source inserts using a query expression can defer to the source to perform the entire insert atomically with source controlled batching. + See maxPreparedInsertBatchSize on any JDBC translator supporting the InsertWithIterator capability. <LI><B>SHOW Statement</B> - added client handling for the SHOW statement to retrieve query plan information and see parameter values. + <LI><B>User Identity</B> - Teiid DataSources in JBossAS connecting to a local Teiid instance can preserve the thread's user identity, rather than requiring + a new authentication. See TeiidDataSource.setPassthroughAuthentication. </UL> <h2><a name="Compatibility">Compatibility Issues</a></h2> Modified: trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCExecutionFactory.java =================================================================== --- trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCExecutionFactory.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCExecutionFactory.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -139,6 +139,7 @@ private boolean trimStrings; private boolean useCommentsInSourceQuery; private String version; + private int maxInsertBatchSize = 2048; private AtomicBoolean initialConnection = new AtomicBoolean(true); @@ -440,6 +441,11 @@ } @Override + public boolean supportsInsertWithIterator() { + return super.supportsBulkUpdate(); + } + + @Override public boolean supportsBatchedUpdates() { return true; } @@ -462,9 +468,25 @@ @Override public boolean supportsInsertWithQueryExpression() { return true; - } + } /** + * Get the max number of inserts to perform in one batch. + * @return + */ + @TranslatorProperty(display="Max Prepared Insert Batch Size", description="The max size of a prepared insert batch. Default 2048.", advanced=true) + public int getMaxPreparedInsertBatchSize() { + return maxInsertBatchSize; + } + + public void setMaxPreparedInsertBatchSize(int maxInsertBatchSize) { + if (maxInsertBatchSize < 1) { + throw new AssertionError("Max prepared batch insert size must be greater than 0"); //$NON-NLS-1$ + } + this.maxInsertBatchSize = maxInsertBatchSize; + } + + /** * Gets the database calendar. This will be set to the time zone * specified by the property {@link JDBCPropertyNames#DATABASE_TIME_ZONE}, or * the local time zone if none is specified. Modified: trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCUpdateExecution.java =================================================================== --- trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCUpdateExecution.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/JDBCUpdateExecution.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -26,14 +26,17 @@ import java.sql.PreparedStatement; import java.sql.SQLException; import java.util.ArrayList; +import java.util.Iterator; import java.util.List; import org.teiid.language.BatchedUpdates; import org.teiid.language.Command; +import org.teiid.language.Insert; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Literal; -import org.teiid.translator.TranslatorException; import org.teiid.translator.DataNotAvailableException; import org.teiid.translator.ExecutionContext; +import org.teiid.translator.TranslatorException; import org.teiid.translator.UpdateExecution; @@ -163,8 +166,46 @@ int updateCount = 0; if (!translatedComm.isPrepared()) { updateCount = getStatement().executeUpdate(sql); + addStatementWarnings(); } else { PreparedStatement pstatement = getPreparedStatement(sql); + + if (command instanceof Insert) { + Insert insert = (Insert)command; + if (insert.getValueSource() instanceof IteratorValueSource) { + commitType = getAutoCommit(translatedComm); + if (commitType) { + connection.setAutoCommit(false); + } + + IteratorValueSource<List<Object>> ivs = (IteratorValueSource)insert.getValueSource(); + List<Object>[] values = new List[ivs.getColumnCount()]; + for (int i = 0; i < ivs.getColumnCount(); i++) { + values[i] = new ArrayList<Object>(); + Literal literal = new Literal(values[i], insert.getColumns().get(i).getType()); + literal.setMultiValued(true); + translatedComm.getPreparedValues().add(literal); + } + Iterator<List<Object>> i = ivs.getIterator(); + int maxBatchSize = this.executionFactory.getMaxPreparedInsertBatchSize(); + while (i.hasNext()) { + int batchSize = 0; + while (i.hasNext() && batchSize++ < maxBatchSize) { + List<Object> next = i.next(); + for (int j = 0; j < ivs.getColumnCount(); j++) { + values[j].add(next.get(j)); + } + } + updateCount += executePreparedBatch(translatedComm, pstatement, batchSize); + for (int j = 0; j < ivs.getColumnCount(); j++) { + values[j].clear(); + } + } + succeeded = true; + return new int[updateCount]; + } + } + int rowCount = 1; for (int i = 0; i< translatedComm.getPreparedValues().size(); i++) { Literal paramValue = (Literal)translatedComm.getPreparedValues().get(i); @@ -179,19 +220,9 @@ connection.setAutoCommit(false); } } - bindPreparedStatementValues(pstatement, translatedComm, rowCount); - if (rowCount > 1) { - int[] results = pstatement.executeBatch(); - - for (int i=0; i<results.length; i++) { - updateCount += results[i]; - } - succeeded = true; - } else { - updateCount = pstatement.executeUpdate(); - } + updateCount = executePreparedBatch(translatedComm, pstatement, rowCount); + succeeded = true; } - addStatementWarnings(); return new int[] {updateCount}; } catch (SQLException err) { throw new JDBCExecutionException(err, translatedComm); @@ -202,6 +233,23 @@ } } + private int executePreparedBatch(TranslatedCommand translatedComm, PreparedStatement pstatement, int rowCount) + throws SQLException { + bindPreparedStatementValues(pstatement, translatedComm, rowCount); + int updateCount = 0; + if (rowCount > 1) { + int[] results = pstatement.executeBatch(); + + for (int i=0; i<results.length; i++) { + updateCount += results[i]; + } + } else { + updateCount = pstatement.executeUpdate(); + } + addStatementWarnings(); + return updateCount; + } + /** * @param command * @return Modified: trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/TranslatedCommand.java =================================================================== --- trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/TranslatedCommand.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/TranslatedCommand.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -27,6 +27,8 @@ import java.util.List; import org.teiid.language.Command; +import org.teiid.language.Insert; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Literal; import org.teiid.language.visitor.CollectorVisitor; import org.teiid.translator.TranslatorException; @@ -89,6 +91,10 @@ return true; } } + if (command instanceof Insert) { + Insert insert = (Insert)command; + return insert.getValueSource() instanceof IteratorValueSource<?>; + } return false; } Modified: trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/modeshape/ModeShapeExecutionFactory.java =================================================================== --- trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/modeshape/ModeShapeExecutionFactory.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/connectors/translator-jdbc/src/main/java/org/teiid/translator/jdbc/modeshape/ModeShapeExecutionFactory.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -30,7 +30,6 @@ import java.util.List; import org.teiid.language.ColumnReference; -import org.teiid.language.Command; import org.teiid.language.Expression; import org.teiid.language.Function; import org.teiid.language.LanguageObject; @@ -39,7 +38,6 @@ import org.teiid.translator.ExecutionContext; import org.teiid.translator.Translator; import org.teiid.translator.TranslatorException; -import org.teiid.translator.TranslatorProperty; import org.teiid.translator.jdbc.ConvertModifier; import org.teiid.translator.jdbc.FunctionModifier; import org.teiid.translator.jdbc.JDBCExecutionFactory; @@ -52,8 +50,10 @@ @Translator(name="modeshape") public class ModeShapeExecutionFactory extends JDBCExecutionFactory { - private String version = "2.0"; //$NON-NLS-1$ - + public ModeShapeExecutionFactory() { + setDatabaseVersion("2.0"); //$NON-NLS-1$ + } + @Override public void start() throws TranslatorException { super.start(); @@ -160,16 +160,7 @@ return "FALSE"; //$NON-NLS-1$ } - - @Override - public List<?> translateCommand(Command command, ExecutionContext context) { - return super.translateCommand(command, context); - } - - - - @Override public String translateLiteralDate(Date dateValue) { return "DATE '" + formatDateValue(dateValue) + "'"; //$NON-NLS-1$//$NON-NLS-2$ } @@ -201,15 +192,6 @@ } - @TranslatorProperty(description= "ModeShape Repository Version") - public String getDatabaseVersion() { - return this.version; - } - - public void setDatabaseVersion(String version) { - this.version = version; - } - @Override public boolean useBindVariables() { return false; Modified: trunk/connectors/translator-jdbc/src/test/java/org/teiid/translator/jdbc/TestJDBCUpdateExecution.java =================================================================== --- trunk/connectors/translator-jdbc/src/test/java/org/teiid/translator/jdbc/TestJDBCUpdateExecution.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/connectors/translator-jdbc/src/test/java/org/teiid/translator/jdbc/TestJDBCUpdateExecution.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -24,13 +24,16 @@ import java.sql.Connection; import java.sql.PreparedStatement; +import java.util.ArrayList; import java.util.Arrays; +import java.util.List; import org.junit.Test; import org.mockito.Mockito; import org.teiid.language.Command; import org.teiid.language.ExpressionValueSource; import org.teiid.language.Insert; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Literal; import org.teiid.translator.ExecutionContext; @@ -58,4 +61,23 @@ Mockito.verify(p, Mockito.times(2)).addBatch(); } + @Test public void testInsertIteratorUpdate() throws Exception { + Insert command = (Insert)TranslationHelper.helpTranslate(TranslationHelper.BQT_VDB, "insert into BQT1.SmallA (IntKey, IntNum) values (1, 2)"); //$NON-NLS-1$ + List<List<Integer>> values = new ArrayList<List<Integer>>(); + values.add(Arrays.asList(1, 2)); + values.add(Arrays.asList(2, 3)); + command.setValueSource(new IteratorValueSource(values.iterator(), 2)); + + Connection connection = Mockito.mock(Connection.class); + PreparedStatement p = Mockito.mock(PreparedStatement.class); + Mockito.stub(p.executeBatch()).toReturn(new int [] {1, 1}); + Mockito.stub(connection.prepareStatement("INSERT INTO SmallA (IntKey, IntNum) VALUES (?, ?)")).toReturn(p); //$NON-NLS-1$ + + JDBCExecutionFactory config = new JDBCExecutionFactory(); + + JDBCUpdateExecution updateExecution = new JDBCUpdateExecution(command, connection, Mockito.mock(ExecutionContext.class), config); + updateExecution.execute(); + Mockito.verify(p, Mockito.times(2)).addBatch(); + } + } Modified: trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml =================================================================== --- trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml 2010-07-29 15:18:33 UTC (rev 2391) @@ -466,8 +466,8 @@ <para>Each <code>Insert</code> will have a single <code>NamedTable</code> specifying the table being inserted into. It will also has a list of <code>ColumnReference</code> specifying the columns of the <code>NamedTable</code> that are being inserted into. It also has - <code>InsertValueSource</code>, which will either be a list of - <code>Expression(ExpressionValueSource)</code> or <code>QueryExpression</code></para> + <code>InsertValueSource</code>, which will be a list of + Expressions (<code>ExpressionValueSource</code>), or a <code>QueryExpression</code>, or an Iterator (<code>IteratorValueSource</code>)</para> </sect3> <sect3> @@ -1245,7 +1245,7 @@ </row> <row> <entry> - <para>supportsBulkUpdate</para> + <para>BulkUpdate</para> </entry> <entry> <para/> @@ -1254,6 +1254,17 @@ <para>Translator supports updates with multiple value sets</para> </entry> </row> + <row> + <entry> + <para>InsertWithIterator</para> + </entry> + <entry> + <para/> + </entry> + <entry> + <para>Translator supports inserts with an iterator of values. The values would typically be from an evaluated QueryExpression.</para> + </entry> + </row> </tbody> </tgroup> </table> Modified: trunk/documentation/reference/src/main/docbook/en-US/content/translators.xml =================================================================== --- trunk/documentation/reference/src/main/docbook/en-US/content/translators.xml 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/documentation/reference/src/main/docbook/en-US/content/translators.xml 2010-07-29 15:18:33 UTC (rev 2391) @@ -196,6 +196,11 @@ <entry>This will embed a /*comment*/ leading comment with session/request id in source SQL query for informational purposes</entry> <entry>false</entry> </row> +<row> +<entry>MaxPreparedInsertBatchSize</entry> +<entry>The max size of a prepared insert batch.</entry> +<entry>2048</entry> +</row> </tbody> </tgroup> </table> Modified: trunk/engine/src/main/java/org/teiid/common/buffer/AbstractTupleSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/AbstractTupleSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/common/buffer/AbstractTupleSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -73,6 +73,8 @@ protected abstract TupleBatch getBatch(int row) throws TeiidComponentException, TeiidProcessingException; + protected abstract int available(); + @Override public void closeSource() { batch = null; Modified: trunk/engine/src/main/java/org/teiid/common/buffer/STree.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -35,10 +35,8 @@ /** * Self balancing search tree using skip list like logic - * This has similar performance similar to a B+/-Tree - * - * TODO: reserve additional memory for delete/update operations - * TODO: double link to support desc key access + * This has similar performance similar to a B+/-Tree, + * but with fewer updates. */ @SuppressWarnings("unchecked") public class STree { @@ -67,7 +65,6 @@ int keyLength, String[] types) { randomSeed = seedGenerator.nextInt() | 0x00000100; // ensure nonzero - randomSeed = 1; this.keyManager = manager; this.leafManager = leafManager; this.comparator = comparator; @@ -327,6 +324,10 @@ return oldSize; } + public int getHeight() { + return header.length; + } + @Override public String toString() { StringBuffer result = new StringBuffer(); Modified: trunk/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -28,11 +28,12 @@ import org.teiid.common.buffer.SPage.SearchResult; import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidProcessingException; /** * Implements intelligent browsing over a {@link STree} */ -public class TupleBrowser { +public class TupleBrowser implements TupleSource { private final STree tree; @@ -132,12 +133,9 @@ return result; } - /** - * Returns the next tuple or null if there are no more results. - * @return - * @throws TeiidComponentException - */ - public List<?> next() throws TeiidComponentException { + @Override + public List<?> nextTuple() throws TeiidComponentException, + TeiidProcessingException { for (;;) { //first check for value iteration if (valueSet != null) { @@ -237,4 +235,10 @@ public void removed() { index-=getOffset(); } + + @Override + public void closeSource() { + + } + } \ No newline at end of file Modified: trunk/engine/src/main/java/org/teiid/common/buffer/TupleBuffer.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/TupleBuffer.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/common/buffer/TupleBuffer.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -268,14 +268,12 @@ * @return */ public IndexedTupleSource createIndexedTupleSource(final boolean singleUse) { + if (singleUse) { + setForwardOnly(true); + } return new AbstractTupleSource() { @Override - public List<? extends Expression> getSchema() { - return (List<? extends Expression>) schema; - } - - @Override protected List<?> finalRow() throws BlockedException { if(isFinal) { return null; Modified: trunk/engine/src/main/java/org/teiid/common/buffer/TupleSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/TupleSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/common/buffer/TupleSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -26,7 +26,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; -import org.teiid.query.sql.symbol.Expression; /** @@ -37,12 +36,6 @@ public interface TupleSource { /** - * Returns the List of ElementSymbol describing the Tuple Source - * @return the List of elements describing the Tuple Source - */ - List<? extends Expression> getSchema(); - - /** * Returns the next tuple * @return the next tuple (a List object), or <code>null</code> if * there are no more tuples. @@ -58,10 +51,4 @@ */ void closeSource(); - /** - * Returns an estimate of the number of rows that can be read without blocking. - * @return - */ - int available(); - } Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -96,7 +96,7 @@ tgtCaps.setCapabilitySupport(Capability.QUERY_ORDERBY_UNRELATED, srcCaps.supportsOrderByUnrelated()); tgtCaps.setCapabilitySupport(Capability.QUERY_AGGREGATES_ENHANCED_NUMERIC, srcCaps.supportsAggregatesEnhancedNumeric()); tgtCaps.setCapabilitySupport(Capability.QUERY_ORDERBY_NULL_ORDERING, srcCaps.supportsOrderByNullOrdering()); - + tgtCaps.setCapabilitySupport(Capability.INSERT_WITH_ITERATOR, srcCaps.supportsInsertWithIterator()); List functions = srcCaps.getSupportedFunctions(); if(functions != null && functions.size() > 0) { Iterator iter = functions.iterator(); Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -25,10 +25,13 @@ import java.util.ArrayList; import java.util.Iterator; import java.util.List; +import java.util.NoSuchElementException; import org.teiid.api.exception.query.QueryMetadataException; import org.teiid.client.metadata.ParameterInfo; +import org.teiid.common.buffer.TupleSource; import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidException; import org.teiid.core.TeiidRuntimeException; import org.teiid.language.AggregateFunction; import org.teiid.language.AndOr; @@ -44,6 +47,7 @@ import org.teiid.language.In; import org.teiid.language.InsertValueSource; import org.teiid.language.IsNull; +import org.teiid.language.IteratorValueSource; import org.teiid.language.Join; import org.teiid.language.Like; import org.teiid.language.Literal; @@ -540,6 +544,40 @@ InsertValueSource valueSource = null; if (insert.getQueryExpression() != null) { valueSource = translate(insert.getQueryExpression()); + } else if (insert.getTupleSource() != null) { + final TupleSource ts = insert.getTupleSource(); + valueSource = new IteratorValueSource<List<?>>(new Iterator<List<?>>() { + + List<?> next; + + @Override + public void remove() { + throw new UnsupportedOperationException(); + } + + @Override + public List<?> next() { + if (hasNext()) { + List<?> result = next; + next = null; + return result; + } + throw new NoSuchElementException(); + } + + @Override + public boolean hasNext() { + if (next != null) { + return true; + } + try { + next = ts.nextTuple(); + } catch (TeiidException e) { + throw new TeiidRuntimeException(e); + } + return next != null; + } + }, elements.size()); } else { // This is for the simple one row insert. List values = insert.getValues(); Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierManagerImpl.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -307,7 +307,7 @@ break; } } - return new CollectionTupleSource(rows.iterator(), command.getProjectedSymbols()); + return new CollectionTupleSource(rows.iterator()); } private List<Schema> getVisibleSchemas(VDBMetaData vdb, CompositeMetadataStore metadata) { Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierTupleSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierTupleSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/DataTierTupleSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -299,12 +299,4 @@ return this.aqr.isTransactional(); } - @Override - public int available() { - if (this.arm == null) { - return 0; - } - return this.arm.getResults().length - index; - } - } Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/PreparedStatementRequest.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/PreparedStatementRequest.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/PreparedStatementRequest.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -41,7 +41,7 @@ import org.teiid.query.QueryPlugin; import org.teiid.query.eval.Evaluator; import org.teiid.query.metadata.QueryMetadataInterface; -import org.teiid.query.optimizer.batch.BatchedUpdatePlanner; +import org.teiid.query.optimizer.BatchedUpdatePlanner; import org.teiid.query.optimizer.capabilities.SourceCapabilities; import org.teiid.query.processor.ProcessorPlan; import org.teiid.query.processor.relational.AccessNode; Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -28,7 +28,6 @@ import java.util.concurrent.atomic.AtomicInteger; import org.teiid.cache.Cache; -import org.teiid.cache.CacheConfiguration; import org.teiid.cache.CacheFactory; import org.teiid.cache.DefaultCache; import org.teiid.cache.DefaultCacheFactory; Modified: trunk/engine/src/main/java/org/teiid/query/function/metadata/FunctionMethod.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/function/metadata/FunctionMethod.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/function/metadata/FunctionMethod.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -69,7 +69,7 @@ /* * always -> normal deterministic functions - * server lifetime -> lookup (however lookup values can be flushed at any time) + * vdb -> lookup (however lookup values can be flushed at any time), current_database * session -> env, user * command -> command payload * never -> rand, etc. @@ -83,7 +83,7 @@ * processing time. */ public static final int DETERMINISTIC = 0; - public static final int SERVER_DETERMINISTIC = 1; + public static final int VDB_DETERMINISTIC = 1; public static final int USER_DETERMINISTIC = 2; public static final int SESSION_DETERMINISTIC = 3; public static final int COMMAND_DETERMINISTIC = 4; Modified: trunk/engine/src/main/java/org/teiid/query/function/source/SystemSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/function/source/SystemSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/function/source/SystemSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -762,7 +762,7 @@ new FunctionParameter("keyelement", DataTypeManager.DefaultDataTypes.STRING, QueryPlugin.Util.getString("SystemSource.Lookup_arg3")), //$NON-NLS-1$ //$NON-NLS-2$ new FunctionParameter("keyvalue", keyValueType, QueryPlugin.Util.getString("SystemSource.Lookup_arg4")), //$NON-NLS-1$ //$NON-NLS-2$ }, - new FunctionParameter("result", DataTypeManager.DefaultDataTypes.OBJECT, QueryPlugin.Util.getString("SystemSource.Lookup_result")), true, FunctionMethod.SERVER_DETERMINISTIC ) ); //$NON-NLS-1$ //$NON-NLS-2$ + new FunctionParameter("result", DataTypeManager.DefaultDataTypes.OBJECT, QueryPlugin.Util.getString("SystemSource.Lookup_result")), true, FunctionMethod.VDB_DETERMINISTIC ) ); //$NON-NLS-1$ //$NON-NLS-2$ } } @@ -773,11 +773,9 @@ } private void addCurrentDatabaseFunction() { - // this function is actually session_deterministic, however for caching purposes it has been set deterministic - // as caching is already scoped to the VDB. functions.add( new FunctionMethod("current_database", QueryPlugin.Util.getString("SystemSource.current_database_desc"), MISCELLANEOUS, FunctionMethod.CANNOT_PUSHDOWN, FUNCTION_CLASS, "current_database", null, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ - new FunctionParameter("result", DataTypeManager.DefaultDataTypes.STRING, QueryPlugin.Util.getString("current_database_result")), false, FunctionMethod.DETERMINISTIC) ); //$NON-NLS-1$ //$NON-NLS-2$ + new FunctionParameter("result", DataTypeManager.DefaultDataTypes.STRING, QueryPlugin.Util.getString("current_database_result")), false, FunctionMethod.VDB_DETERMINISTIC) ); //$NON-NLS-1$ //$NON-NLS-2$ } private void addEnvFunction() { Modified: trunk/engine/src/main/java/org/teiid/query/metadata/TempCapabilitiesFinder.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/metadata/TempCapabilitiesFinder.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/metadata/TempCapabilitiesFinder.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -43,7 +43,7 @@ if (TempMetadataAdapter.TEMP_MODEL.getID().equals(modelName)) { if (tempCaps == null) { tempCaps = new BasicSourceCapabilities(); - tempCaps.setCapabilitySupport(Capability.BULK_UPDATE, true); + tempCaps.setCapabilitySupport(Capability.INSERT_WITH_ITERATOR, true); tempCaps.setCapabilitySupport(Capability.QUERY_ORDERBY, true); tempCaps.setCapabilitySupport(Capability.CRITERIA_NOT, true); tempCaps.setCapabilitySupport(Capability.CRITERIA_IN, true); Copied: trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java (from rev 2386, trunk/engine/src/main/java/org/teiid/query/optimizer/batch/BatchedUpdatePlanner.java) =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java (rev 0) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -0,0 +1,236 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + +package org.teiid.query.optimizer; + +import java.util.ArrayList; +import java.util.List; + +import org.teiid.api.exception.query.QueryMetadataException; +import org.teiid.api.exception.query.QueryPlannerException; +import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidRuntimeException; +import org.teiid.core.id.IDGenerator; +import org.teiid.core.id.IntegerID; +import org.teiid.query.analysis.AnalysisRecord; +import org.teiid.query.execution.QueryExecPlugin; +import org.teiid.query.metadata.QueryMetadataInterface; +import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; +import org.teiid.query.optimizer.capabilities.SourceCapabilities; +import org.teiid.query.optimizer.capabilities.SourceCapabilities.Capability; +import org.teiid.query.optimizer.relational.rules.CapabilitiesUtil; +import org.teiid.query.processor.ProcessorPlan; +import org.teiid.query.processor.batch.BatchedUpdatePlan; +import org.teiid.query.processor.relational.BatchedUpdateNode; +import org.teiid.query.processor.relational.ProjectNode; +import org.teiid.query.processor.relational.RelationalPlan; +import org.teiid.query.sql.lang.BatchedUpdateCommand; +import org.teiid.query.sql.lang.Command; +import org.teiid.query.sql.lang.Delete; +import org.teiid.query.sql.lang.Insert; +import org.teiid.query.sql.lang.Update; +import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.util.VariableContext; +import org.teiid.query.sql.visitor.EvaluatableVisitor; +import org.teiid.query.util.CommandContext; + + + +/** + * Planner for BatchedUpdateCommands + * @since 4.2 + */ +public class BatchedUpdatePlanner implements CommandPlanner { + + /** + * Optimizes batched updates by batching all contiguous commands that relate to the same physical model. + * For example, for the following batch of commands: + * <br/> + * <ol> + * <li>1. INSERT INTO physicalModel.myPhysical ...</li> + * <li>2. UPDATE physicalModel.myPhysical ... </li> + * <li>3. DELETE FROM virtualmodel.myVirtual ... </li> + * <li>4. UPDATE virtualmodel.myVirtual ... </li> + * <li>5. UPDATE physicalModel.myOtherPhysical ...</li> + * <li>6. INSERT INTO physicalModel.myOtherPhysical ... <li> + * <li>7. DELETE FROM physicalModel.myOtherPhysical ...</li> + * <li>8. INSERT INTO physicalModel.myPhysical ... </li> + * <li>9. INSERT INTO physicalModel.myPhysical ... </li> + * <li>10. INSERT INTO physicalModel.myPhysical ... </li> + * <li>11. INSERT INTO physicalModel.myPhysical ... </li> + * <li>12. INSERT INTO physicalModel.myPhysical ... </li> + * </ol> + * <br/> this implementation will batch as follows: (1,2), (5, 6, 7), (8 thru 12). + * The remaining commands/plans will be executed individually. + * @see org.teiid.query.optimizer.CommandPlanner#optimize(Command, org.teiid.core.id.IDGenerator, org.teiid.query.metadata.QueryMetadataInterface, org.teiid.query.optimizer.capabilities.CapabilitiesFinder, org.teiid.query.analysis.AnalysisRecord, CommandContext) + * @since 4.2 + */ + public ProcessorPlan optimize(Command command, + IDGenerator idGenerator, + QueryMetadataInterface metadata, + CapabilitiesFinder capFinder, + AnalysisRecord analysisRecord, CommandContext context) + throws QueryPlannerException, QueryMetadataException, TeiidComponentException { + BatchedUpdateCommand batchedUpdateCommand = (BatchedUpdateCommand)command; + List childPlans = new ArrayList(batchedUpdateCommand.getUpdateCommands().size()); + List updateCommands = batchedUpdateCommand.getUpdateCommands(); + int numCommands = updateCommands.size(); + List<VariableContext> allContexts = batchedUpdateCommand.getVariableContexts(); + List<VariableContext> planContexts = null; + if (allContexts != null) { + planContexts = new ArrayList<VariableContext>(allContexts.size()); + } + for (int commandIndex = 0; commandIndex < numCommands; commandIndex++) { + // Potentially the first command of a batch + Command updateCommand = (Command)updateCommands.get(commandIndex); + boolean commandWasBatched = false; + // If this command can be placed in a batch + if (isEligibleForBatching(updateCommand, metadata)) { + // Get the model ID. Subsequent and contiguous commands that update a group in this model are candidates for this batch + Object batchModelID = metadata.getModelID(getUpdatedGroup(updateCommand).getMetadataID()); + String modelName = metadata.getFullName(batchModelID); + SourceCapabilities caps = capFinder.findCapabilities(modelName); + // Only attempt batching if the source supports batching + if (caps.supportsCapability(Capability.BATCHED_UPDATES)) { + // Start a new batch + List<Command> batch = new ArrayList<Command>(); + List<VariableContext> contexts = new ArrayList<VariableContext>(); + List<Boolean> shouldEvaluate = new ArrayList<Boolean>(); + // This is the first command in a potential batch, so add it to the batch + batch.add(updateCommand); + if (allContexts != null) { + contexts.add(allContexts.get(commandIndex)); + shouldEvaluate.add(Boolean.TRUE); + } else { + shouldEvaluate.add(EvaluatableVisitor.needsProcessingEvaluation(updateCommand)); + } + // Find out if there are other commands called on the same physical model + // immediately and contiguously after this one + batchLoop: for (int batchIndex = commandIndex+1; batchIndex < numCommands; batchIndex++) { + Command batchingCandidate = (Command)updateCommands.get(batchIndex); + // If this command updates the same model, and is eligible for batching, add it to the batch + if (canBeAddedToBatch(batchingCandidate, batchModelID, metadata, capFinder)) { + batch.add(batchingCandidate); + if (allContexts != null) { + contexts.add(allContexts.get(batchIndex)); + shouldEvaluate.add(Boolean.TRUE); + } else { + shouldEvaluate.add(EvaluatableVisitor.needsProcessingEvaluation(batchingCandidate)); + } + } else { // Otherwise, stop batching at this point. The next command may well be the start of a new batch + break batchLoop; + } + } + // If two or more contiguous commands made on the same model were found, then batch them + if (batch.size() > 1) { + ProjectNode projectNode = new ProjectNode(((IntegerID)idGenerator.create()).getValue()); + // Create a BatchedUpdateNode that creates a batched request for the connector + BatchedUpdateNode batchNode = new BatchedUpdateNode(((IntegerID)idGenerator.create()).getValue(), + batch, contexts, shouldEvaluate, + modelName); + List symbols = batchedUpdateCommand.getProjectedSymbols(); + projectNode.setSelectSymbols(symbols); + projectNode.setElements(symbols); + + batchNode.setElements(symbols); + + projectNode.addChild(batchNode); + // Add a new RelationalPlan that represents the plan for this batch. + childPlans.add(new RelationalPlan(projectNode)); + if (planContexts != null) { + planContexts.add(new VariableContext()); + } + // Skip those commands that were added to this batch + commandIndex += batch.size() - 1; + commandWasBatched = true; + } + } + } + if (!commandWasBatched) { // If the command wasn't batched, just add the plan for this command to the list of plans + Command cmd = (Command)batchedUpdateCommand.getUpdateCommands().get(commandIndex); + ProcessorPlan plan = cmd.getProcessorPlan(); + if (plan == null) { + plan = QueryOptimizer.optimizePlan(cmd, metadata, idGenerator, capFinder, analysisRecord, context); + } + childPlans.add(plan); + if (allContexts != null) { + planContexts.add(allContexts.get(commandIndex)); + } + } + } + return new BatchedUpdatePlan(childPlans, batchedUpdateCommand.getUpdateCommands().size(), planContexts); + } + + /** + * Get the group being updated by the update command + * @param command an INSERT, UPDATE, DELETE or SELECT INTO command + * @return the group being updated + * @since 4.2 + */ + public static GroupSymbol getUpdatedGroup(Command command) { + int type = command.getType(); + if (type == Command.TYPE_INSERT) { + return ((Insert)command).getGroup(); + } else if (type == Command.TYPE_UPDATE) { + return ((Update)command).getGroup(); + } else if (type == Command.TYPE_DELETE) { + return ((Delete)command).getGroup(); + } + throw new TeiidRuntimeException(QueryExecPlugin.Util.getString("BatchedUpdatePlanner.unrecognized_command", command)); //$NON-NLS-1$ + } + + /** + * Returns whether a command can be placed in a connector batch + * @param command an update command + * @param metadata + * @return true if this command can be added to a batch; false otherwise + * @throws QueryMetadataException + * @throws TeiidComponentException + * @since 4.2 + */ + public static boolean isEligibleForBatching(Command command, QueryMetadataInterface metadata) throws QueryMetadataException, TeiidComponentException { + // If the command updates a physical group, it's eligible + return !metadata.isVirtualGroup(getUpdatedGroup(command).getMetadataID()); + } + + /** + * Returns whether a command can be placed in a given batch + * @param command an update command + * @param batchModelID the model ID for the batch concerned + * @param metadata + * @param capFinder + * @return true if this command can be place in a batch associated with the model ID; false otherwise + * @throws QueryMetadataException + * @throws TeiidComponentException + * @since 4.2 + */ + private static boolean canBeAddedToBatch(Command command, Object batchModelID, QueryMetadataInterface metadata, CapabilitiesFinder capFinder) throws QueryMetadataException, TeiidComponentException { + // If it's eligible ... + if (isEligibleForBatching(command, metadata)) { + Object modelID = metadata.getModelID(getUpdatedGroup(command).getMetadataID()); + + return CapabilitiesUtil.isSameConnector(modelID, batchModelID, metadata, capFinder); + } + return false; + } + +} Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -36,7 +36,6 @@ import org.teiid.query.metadata.TempCapabilitiesFinder; import org.teiid.query.metadata.TempMetadataAdapter; import org.teiid.query.metadata.TempMetadataStore; -import org.teiid.query.optimizer.batch.BatchedUpdatePlanner; import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; import org.teiid.query.optimizer.proc.ProcedurePlanner; import org.teiid.query.optimizer.relational.RelationalPlanner; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -317,7 +317,8 @@ * @since 6.0.0 indicates support for where all */ REQUIRES_CRITERIA, - INSERT_WITH_QUERYEXPRESSION + INSERT_WITH_QUERYEXPRESSION, + INSERT_WITH_ITERATOR } public enum Scope { Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -192,8 +192,15 @@ pinode.setModelName(modelName); processNode = pinode; SourceCapabilities caps = capFinder.findCapabilities(modelName); - pinode.setDoBatching(caps.supportsCapability(Capability.BATCHED_UPDATES)); - pinode.setDoBulkInsert(caps.supportsCapability(Capability.BULK_UPDATE)); + if (caps.supportsCapability(Capability.INSERT_WITH_ITERATOR)) { + pinode.setMode(org.teiid.query.processor.relational.ProjectIntoNode.Mode.ITERATOR); + } else if (caps.supportsCapability(Capability.BULK_UPDATE)) { + pinode.setMode(org.teiid.query.processor.relational.ProjectIntoNode.Mode.BULK); + } else if (caps.supportsCapability(Capability.BATCHED_UPDATES)) { + pinode.setMode(org.teiid.query.processor.relational.ProjectIntoNode.Mode.BATCH); + } else { + pinode.setMode(org.teiid.query.processor.relational.ProjectIntoNode.Mode.SINGLE); + } } } catch(QueryMetadataException e) { throw new TeiidComponentException(e); Modified: trunk/engine/src/main/java/org/teiid/query/processor/BatchIterator.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/BatchIterator.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/BatchIterator.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -31,7 +31,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.query.processor.BatchCollector.BatchProducer; -import org.teiid.query.sql.symbol.Expression; /** @@ -87,7 +86,6 @@ return tuple; } - @Override public int available() { if (batch != null && batch.containsRow(getCurrentIndex())) { return batch.getEndRow() - getCurrentIndex() + 1; @@ -95,11 +93,6 @@ return 0; } - @Override - public List<? extends Expression> getSchema() { - return source.getOutputElements(); - } - public void setBuffer(TupleBuffer buffer, boolean saveOnMark) { this.buffer = buffer; this.saveOnMark = saveOnMark; Modified: trunk/engine/src/main/java/org/teiid/query/processor/CollectionTupleSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/CollectionTupleSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/CollectionTupleSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -28,27 +28,22 @@ import java.util.List; import org.teiid.common.buffer.TupleSource; -import org.teiid.query.sql.lang.Command; -import org.teiid.query.sql.symbol.SingleElementSymbol; public class CollectionTupleSource implements TupleSource { private Iterator<? extends List<?>> tuples; - private List<? extends SingleElementSymbol> schema; public static CollectionTupleSource createUpdateCountTupleSource(int count) { - return new CollectionTupleSource(Arrays.asList(Arrays.asList(count)).iterator(), Command.getUpdateCommandSymbol()); + return new CollectionTupleSource(Arrays.asList(Arrays.asList(count)).iterator()); } - public static CollectionTupleSource createNullTupleSource(List<SingleElementSymbol> schema) { - return new CollectionTupleSource(new ArrayList<List<Object>>(0).iterator(), schema); + public static CollectionTupleSource createNullTupleSource() { + return new CollectionTupleSource(new ArrayList<List<Object>>(0).iterator()); } - public CollectionTupleSource(Iterator<? extends List<?>> tuples, - List<? extends SingleElementSymbol> schema) { + public CollectionTupleSource(Iterator<? extends List<?>> tuples) { this.tuples = tuples; - this.schema = schema; } @Override @@ -60,20 +55,8 @@ } @Override - public List<? extends SingleElementSymbol> getSchema() { - return schema; - } - - @Override public void closeSource() { } - @Override - public int available() { - if (tuples.hasNext()) { - return 1; - } - return 0; - } } \ No newline at end of file Modified: trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/proc/ProcedurePlan.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -65,7 +65,6 @@ import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.util.VariableContext; import org.teiid.query.tempdata.TempTableStore; -import org.teiid.query.tempdata.TempTableStore; import org.teiid.query.util.CommandContext; import org.teiid.query.util.ErrorMessageKeys; @@ -300,7 +299,7 @@ } if(lastTupleSource == null){ - return CollectionTupleSource.createNullTupleSource(null); + return CollectionTupleSource.createNullTupleSource(); } return lastTupleSource; } @@ -558,7 +557,7 @@ CursorState cursorState = getCursorState(rsKey); // get the schema from the tuple source - List schema = cursorState.ts.getSchema(); + List schema = cursorState.processor.getOutputElements(); return schema; } Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -89,7 +89,7 @@ sortSymbols.add(dependentSetStates.get(i).valueExpression); } DependentValueSource originalVs = (DependentValueSource)dependentNode.getContext().getVariableContext().getGlobalValue(valueSource); - this.sortUtility = new SortUtility(originalVs.getTupleBuffer().createIndexedTupleSource(), sortSymbols, sortDirection, Mode.DUP_REMOVE, dependentNode.getBufferManager(), dependentNode.getConnectionID()); + this.sortUtility = new SortUtility(originalVs.getTupleBuffer().createIndexedTupleSource(), sortSymbols, sortDirection, Mode.DUP_REMOVE, dependentNode.getBufferManager(), dependentNode.getConnectionID(), originalVs.getTupleBuffer().getSchema()); } dvs = new DependentValueSource(sortUtility.sort()); for (SetState setState : dependentSetStates) { Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentValueSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentValueSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentValueSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -65,7 +65,7 @@ IndexedTupleSource its = buffer.createIndexedTupleSource(); int index = 0; if (valueExpression != null) { - index = its.getSchema().indexOf(valueExpression); + index = buffer.getSchema().indexOf(valueExpression); Assertion.assertTrue(index != -1); } return new TupleSourceValueIterator(its, index); @@ -83,10 +83,10 @@ IndexedTupleSource its = buffer.createIndexedTupleSource(); int index = 0; if (valueExpression != null) { - index = its.getSchema().indexOf(valueExpression); + index = buffer.getSchema().indexOf(valueExpression); } Assertion.assertTrue(index != -1); - if (((SingleElementSymbol)its.getSchema().get(index)).getType() == DataTypeManager.DefaultDataClasses.BIG_DECIMAL) { + if (((SingleElementSymbol)buffer.getSchema().get(index)).getType() == DataTypeManager.DefaultDataClasses.BIG_DECIMAL) { result = new TreeSet<Object>(); } else { result = new HashSet<Object>(); Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -306,22 +306,10 @@ } @Override - public List<Expression> getSchema() { - return collectedExpressions; - } - - @Override public void closeSource() { } - @Override - public int available() { - if (sourceBatch != null) { - return sourceBatch.getEndRow() - sourceRow + 1; - } - return 0; - } }; } @@ -334,7 +322,7 @@ } else { this.sortUtility = new SortUtility(getCollectionTupleSource(), sortElements, sortTypes, removeDuplicates?Mode.DUP_REMOVE_SORT:Mode.SORT, getBufferManager(), - getConnectionID()); + getConnectionID(), collectedExpressions); this.phase = SORT; } } Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/ProjectIntoNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -32,7 +32,9 @@ import org.teiid.client.plan.PlanNode; import org.teiid.common.buffer.BlockedException; import org.teiid.common.buffer.TupleBatch; +import org.teiid.common.buffer.TupleBuffer; import org.teiid.common.buffer.TupleSource; +import org.teiid.common.buffer.BufferManager.TupleSourceType; import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.query.sql.lang.BatchedUpdateCommand; @@ -45,6 +47,10 @@ public class ProjectIntoNode extends RelationalNode { + public enum Mode { + BATCH, BULK, ITERATOR, SINGLE + } + private static int REQUEST_CREATION = 1; private static int RESPONSE_PROCESSING = 2; @@ -52,8 +58,7 @@ private GroupSymbol intoGroup; private List intoElements; private String modelName; - private boolean doBatching = false; - private boolean doBulkInsert = false; + private Mode mode; // Processing state private int batchRow = 1; @@ -61,7 +66,9 @@ private int phase = REQUEST_CREATION; private int requestsRegistered = 0; private int tupleSourcesProcessed = 0; + private boolean sourceDone; + private TupleBuffer buffer; private TupleBatch currentBatch; private TupleSource tupleSource; @@ -78,6 +85,7 @@ this.tupleSourcesProcessed = 0; this.requestsRegistered = 0; this.currentBatch=null; + this.sourceDone=false; } public void setIntoGroup(GroupSymbol group) { @@ -107,28 +115,45 @@ checkExitConditions(); - /* If we don't have a batch to work with yet, or - * if this is not the last batch and we've reached the end of this one, - * then get the next batch. + /* If we don't have a batch to work, get the next */ - - if (currentBatch == null || (!currentBatch.getTerminationFlag() && this.batchRow > currentBatch.getEndRow())) { + if (currentBatch == null) { + if (sourceDone) { + phase = RESPONSE_PROCESSING; + break; + } currentBatch = getChildren()[0].nextBatch(); // can throw BlockedException + sourceDone = currentBatch.getTerminationFlag(); this.batchRow = currentBatch.getBeginRow(); //normally we would want to skip a 0 sized batch, but it typically represents the terminal batch //and for implicit temp tables we need to issue an empty insert - if(currentBatch.getRowCount() == 0 && (!this.intoGroup.isImplicitTempGroupSymbol() || !currentBatch.getTerminationFlag())) { - continue; + if(currentBatch.getRowCount() == 0 + && (!currentBatch.getTerminationFlag() || mode != Mode.ITERATOR)) { + currentBatch = null; + continue; } - } else if (currentBatch.getTerminationFlag() && this.batchRow > currentBatch.getEndRow()) { - phase = RESPONSE_PROCESSING; - break; - } + } int batchSize = currentBatch.getRowCount(); - - if (doBulkInsert) { + int requests = 1; + switch (mode) { + case ITERATOR: + if (buffer == null) { + buffer = getBufferManager().createTupleBuffer(intoElements, getConnectionID(), TupleSourceType.PROCESSOR); + } + buffer.addTupleBatch(currentBatch, true); + if (currentBatch.getTerminationFlag() && (buffer.getRowCount() != 0 || intoGroup.isImplicitTempGroupSymbol())) { + Insert insert = new Insert(intoGroup, intoElements, null); + buffer.close(); + insert.setTupleSource(buffer.createIndexedTupleSource(true)); + // Register insert command against source + registerRequest(insert); + } else { + requests = 0; + } + break; + case BULK: //convert to multivalued parameter List<Constant> parameters = new ArrayList<Constant>(intoElements.size()); for (int i = 0; i < intoElements.size(); i++) { @@ -145,26 +170,32 @@ Insert insert = new Insert(intoGroup, intoElements, parameters); // Register insert command against source registerRequest(insert); - } else if (doBatching) { + break; + case BATCH: // Register batched update command against source int endRow = currentBatch.getEndRow(); List rows = new ArrayList(endRow-batchRow); for(int rowNum = batchRow; rowNum <= endRow; rowNum++) { - Insert insert = new Insert( intoGroup, + insert = new Insert( intoGroup, intoElements, convertValuesToConstants(currentBatch.getTuple(rowNum), intoElements)); rows.add( insert ); } registerRequest(new BatchedUpdateCommand( rows )); - } else { + break; + case SINGLE: batchSize = 1; // Register insert command against source // Defect 16036 - submit a new INSERT command to the DataManager. registerRequest(new Insert(intoGroup, intoElements, convertValuesToConstants(currentBatch.getTuple(batchRow), intoElements))); } + this.batchRow += batchSize; - this.requestsRegistered++; + if (batchRow > currentBatch.getEndRow()) { + currentBatch = null; + } + this.requestsRegistered+=requests; } checkExitConditions(); @@ -197,7 +228,10 @@ } private void closeRequest() { - + if (this.buffer != null) { + this.buffer.remove(); + this.buffer = null; + } if (this.tupleSource != null) { tupleSource.closeSource(); this.tupleSource = null; @@ -216,8 +250,7 @@ clonedNode.intoGroup = intoGroup; clonedNode.intoElements = intoElements; clonedNode.modelName = this.modelName; - clonedNode.doBatching = this.doBatching; - clonedNode.doBulkInsert = this.doBulkInsert; + clonedNode.mode = this.mode; return clonedNode; } @@ -242,15 +275,15 @@ constants.add(new Constant(values.get(i),type)); } return constants; - } + } + + public Mode getMode() { + return mode; + } - public void setDoBatching(boolean doBatching) { - this.doBatching = doBatching; - } - - public void setDoBulkInsert(boolean doBulkInsert) { - this.doBulkInsert = doBulkInsert; - } + public void setMode(Mode mode) { + this.mode = mode; + } public boolean isTempGroupInsert() { return intoGroup.isTempGroupSymbol(); Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/RelationalPlan.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/RelationalPlan.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/RelationalPlan.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -34,6 +34,7 @@ import org.teiid.core.TeiidProcessingException; import org.teiid.query.processor.ProcessorDataManager; import org.teiid.query.processor.ProcessorPlan; +import org.teiid.query.processor.relational.ProjectIntoNode.Mode; import org.teiid.query.sql.lang.QueryCommand; import org.teiid.query.util.CommandContext; @@ -157,7 +158,12 @@ } return false; } - if (node instanceof AccessNode) { + if (node instanceof ProjectIntoNode) { + if (((ProjectIntoNode)node).getMode() == Mode.ITERATOR) { + return false; + } + return true; + } else if (node instanceof AccessNode) { return false; } if (transactionalReads) { Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/SortNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/SortNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/SortNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -91,7 +91,7 @@ private void sortPhase() throws BlockedException, TeiidComponentException, TeiidProcessingException { if (this.sortUtility == null) { this.sortUtility = new SortUtility(new BatchIterator(getChildren()[0]), items, this.mode, getBufferManager(), - getConnectionID()); + getConnectionID(), getChildren()[0].getElements()); } this.output = this.sortUtility.sort(); if (this.outputTs == null) { Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -108,12 +108,12 @@ private Collection<List<?>> workingTuples; public SortUtility(TupleSource sourceID, List<OrderByItem> items, Mode mode, BufferManager bufferMgr, - String groupName) { + String groupName, List schema) { this.source = sourceID; this.mode = mode; this.bufferManager = bufferMgr; this.groupName = groupName; - this.schema = this.source.getSchema(); + this.schema = schema; this.schemaSize = bufferManager.getSchemaSize(this.schema); int distinctIndex = items != null? items.size() - 1:0; List<Expression> sortElements = null; @@ -153,8 +153,8 @@ } public SortUtility(TupleSource ts, List expressions, List<Boolean> types, - Mode mode, BufferManager bufferManager, String connectionID) { - this(ts, new OrderBy(expressions, types).getOrderByItems(), mode, bufferManager, connectionID); + Mode mode, BufferManager bufferManager, String connectionID, List schema) { + this(ts, new OrderBy(expressions, types).getOrderByItems(), mode, bufferManager, connectionID, schema); } public boolean isDone() { Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/SortingFilter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/SortingFilter.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/SortingFilter.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -134,7 +134,7 @@ // Sort if (sortUtility == null) { - sortUtility = new SortUtility(collectionBuffer.createIndexedTupleSource(), sortItems, removeDuplicates?Mode.DUP_REMOVE_SORT:Mode.SORT, mgr, groupName); + sortUtility = new SortUtility(collectionBuffer.createIndexedTupleSource(), sortItems, removeDuplicates?Mode.DUP_REMOVE_SORT:Mode.SORT, mgr, groupName, collectionBuffer.getSchema()); } TupleBuffer sorted = sortUtility.sort(); sorted.setForwardOnly(true); Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/SourceState.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/SourceState.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/SourceState.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -187,7 +187,7 @@ ts = new BatchIterator(this.source); } this.sortUtility = new SortUtility(ts, expressions, Collections.nCopies(expressions.size(), OrderBy.ASC), - sortOption == SortOption.SORT_DISTINCT?Mode.DUP_REMOVE_SORT:Mode.SORT, this.source.getBufferManager(), this.source.getConnectionID()); + sortOption == SortOption.SORT_DISTINCT?Mode.DUP_REMOVE_SORT:Mode.SORT, this.source.getBufferManager(), this.source.getConnectionID(), source.getElements()); this.markDistinct(sortOption == SortOption.SORT_DISTINCT && expressions.size() == this.getOuterVals().size()); } this.buffer = sortUtility.sort(); Modified: trunk/engine/src/main/java/org/teiid/query/sql/lang/Insert.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/lang/Insert.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/sql/lang/Insert.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -29,6 +29,7 @@ import java.util.List; import java.util.Map; +import org.teiid.common.buffer.TupleSource; import org.teiid.core.util.EquivalenceUtil; import org.teiid.core.util.HashCodeUtil; import org.teiid.query.sql.LanguageVisitor; @@ -38,6 +39,7 @@ import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; +import org.teiid.query.sql.util.ValueIterator; /** @@ -52,10 +54,12 @@ /** list of column variables, null = all columns */ private List variables = new LinkedList(); - /** List of Expressions, required */ + /** List of Expressions */ private List values = new LinkedList(); private QueryCommand queryExpression; + + private TupleSource tupleSource; // ========================================================================= // C O N S T R U C T O R S @@ -298,5 +302,13 @@ public boolean areResultsCachable() { return false; } + + public void setTupleSource(TupleSource tupleSource) { + this.tupleSource = tupleSource; + } + + public TupleSource getTupleSource() { + return tupleSource; + } } Modified: trunk/engine/src/main/java/org/teiid/query/sql/lang/Query.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/lang/Query.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/sql/lang/Query.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -434,7 +434,7 @@ static boolean areResultsCachable(List projectedSymbols) { for(int i=0; i<projectedSymbols.size(); i++){ SingleElementSymbol projectedSymbol = (SingleElementSymbol)projectedSymbols.get(i); - if(DataTypeManager.isLOB(projectedSymbol.getType())) { + if(DataTypeManager.isLOB(projectedSymbol.getType()) || projectedSymbol.getType() == DataTypeManager.DefaultDataClasses.OBJECT) { return false; } } Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -403,6 +403,9 @@ if ( obj.getQueryExpression() != null ) { parts.add(registerNode(obj.getQueryExpression())); + } else if (obj.getTupleSource() != null) { + parts.add(VALUES); + parts.add(" (...)"); //$NON-NLS-1$ } else { parts.add(VALUES); parts.add(" ("); //$NON-NLS-1$ Modified: trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -37,6 +37,7 @@ import org.teiid.common.buffer.TupleBrowser; import org.teiid.common.buffer.TupleBuffer; import org.teiid.common.buffer.TupleSource; +import org.teiid.common.buffer.BufferManager.BufferReserveMode; import org.teiid.common.buffer.BufferManager.TupleSourceType; import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidException; @@ -65,29 +66,75 @@ */ class TempTable { - private final class QueryTupleSource extends TupleBrowserTupleSource { + private final class InsertUpdateProcessor extends UpdateProcessor { + + private boolean addRowId; + private int[] indexes; + + private InsertUpdateProcessor(TupleSource ts, boolean addRowId, int[] indexes) + throws TeiidComponentException { + super(null, ts); + this.addRowId = addRowId; + this.indexes = indexes; + } + + @Override + protected void tuplePassed(List tuple) throws BlockedException, + TeiidComponentException, TeiidProcessingException { + if (indexes != null) { + List<Object> newTuple = new ArrayList<Object>(columns.size()); + if (addRowId) { + newTuple.add(rowId.getAndIncrement()); + } + for (int i = 0; i < indexes.length; i++) { + if (indexes[i] == -1) { + newTuple.add(null); + } else { + newTuple.add(tuple.get(indexes[i])); + } + } + tuple = newTuple; + } else if (addRowId) { + tuple = new ArrayList<Object>(tuple); + tuple.add(0, rowId.getAndIncrement()); + } + currentTuple = tuple; + insertTuple(tuple); + } + + @Override + protected void undo(List tuple) throws TeiidComponentException, + TeiidProcessingException { + deleteTuple(tuple); + } + } + + private final class QueryTupleSource implements TupleSource { private final Evaluator eval; - private final List<SingleElementSymbol> projectedCols; private final Criteria condition; private final boolean project; private final int[] indexes; + private int reserved; + private TupleBrowser browser; private QueryTupleSource(TupleBrowser browser, Map map, List<SingleElementSymbol> projectedCols, Criteria condition) { - super(browser); + this.browser = browser; this.indexes = RelationalNode.getProjectionIndexes(map, projectedCols); this.eval = new Evaluator(map, null, null); - this.projectedCols = projectedCols; this.condition = condition; this.project = shouldProject(); + reserved = reserveBuffers(); } @Override public List<?> nextTuple() throws TeiidComponentException, TeiidProcessingException { for (;;) { - List<?> next = super.nextTuple(); + List<?> next = browser.nextTuple(); if (next == null) { + bm.releaseBuffers(reserved); + reserved = 0; return null; } if (rowId != null) { @@ -103,6 +150,13 @@ } } + @Override + public void closeSource() { + bm.releaseBuffers(reserved); + reserved = 0; + browser.closeSource(); + } + private boolean shouldProject() { if (indexes.length == getColumns().size()) { for (int i = 0; i < indexes.length; i++) { @@ -115,48 +169,15 @@ return true; } - @Override - public List<? extends Expression> getSchema() { - return projectedCols; - } } - private class TupleBrowserTupleSource implements TupleSource { - private final TupleBrowser browser; - - private TupleBrowserTupleSource(TupleBrowser browser) { - this.browser = browser; - } - - @Override - public List<?> nextTuple() throws TeiidComponentException, - TeiidProcessingException { - return browser.next(); - } - - @Override - public List<? extends Expression> getSchema() { - return columns; - } - - @Override - public void closeSource() { - - } - - @Override - public int available() { - return 0; - } - } - private abstract class UpdateProcessor { private TupleSource ts; protected final Map lookup; protected final Evaluator eval; private final Criteria crit; protected int updateCount = 0; - private List currentTuple; + protected List currentTuple; protected TupleBuffer undoLog; @@ -169,6 +190,7 @@ } int process() throws ExpressionEvaluationException, TeiidComponentException, TeiidProcessingException { + int reserved = reserveBuffers(); boolean success = false; try { while (currentTuple != null || (currentTuple = ts.nextTuple()) != null) { @@ -179,9 +201,12 @@ } currentTuple = null; } + bm.releaseBuffers(reserved); + reserved = 0; success(); success = true; } finally { + bm.releaseBuffers(reserved); if (!success) { TupleSource undoTs = undoLog.createIndexedTupleSource(); List<?> tuple = null; @@ -197,7 +222,7 @@ } return updateCount; } - + @SuppressWarnings("unused") void success() throws TeiidComponentException, ExpressionEvaluationException, TeiidProcessingException {} @@ -218,6 +243,9 @@ private BufferManager bm; private String sessionID; private TempMetadataID tid; + + private int keyBatchSize; + private int leafBatchSize; TempTable(TempMetadataID tid, BufferManager bm, List<ElementSymbol> columns, int primaryKeyLength, String sessionID) { this.tid = tid; @@ -233,7 +261,13 @@ } this.columns = columns; this.sessionID = sessionID; + this.keyBatchSize = bm.getSchemaSize(columns); + this.leafBatchSize = bm.getSchemaSize(columns.subList(0, primaryKeyLength)); } + + private int reserveBuffers() { + return bm.reserveBuffers(leafBatchSize + (tree.getHeight() - 1)*keyBatchSize, BufferReserveMode.WAIT); + } public TupleSource createTupleSource(final List<SingleElementSymbol> projectedCols, final Criteria condition, OrderBy orderBy) throws TeiidComponentException, TeiidProcessingException { Map map = RelationalNode.createLookupMap(getColumns()); @@ -282,21 +316,24 @@ TupleBrowser browser = createTupleBrower(condition, direction); TupleSource ts = new QueryTupleSource(browser, map, projectedCols, condition); - - TupleBuffer tb = null; - if (!orderByUsingIndex && orderBy != null) { - SortUtility sort = new SortUtility(ts, orderBy.getOrderByItems(), Mode.SORT, bm, sessionID); - tb = sort.sort(); - } else { - tb = bm.createTupleBuffer(projectedCols, sessionID, TupleSourceType.PROCESSOR); - List next = null; - while ((next = ts.nextTuple()) != null) { - tb.addTuple(next); + try { + TupleBuffer tb = null; + if (!orderByUsingIndex && orderBy != null) { + SortUtility sort = new SortUtility(ts, orderBy.getOrderByItems(), Mode.SORT, bm, sessionID, projectedCols); + tb = sort.sort(); + } else { + tb = bm.createTupleBuffer(projectedCols, sessionID, TupleSourceType.PROCESSOR); + List next = null; + while ((next = ts.nextTuple()) != null) { + tb.addTuple(next); + } } + tb.close(); + return tb.createIndexedTupleSource(true); + } finally { + //ensure the buffers get released + ts.closeSource(); } - tb.close(); - tb.setForwardOnly(true); - return tb.createIndexedTupleSource(true); } private TupleBrowser createTupleBrower(Criteria condition, boolean direction) throws TeiidComponentException { @@ -353,23 +390,15 @@ return columns; } - public TupleSource insert(List<List<Object>> tuples) throws TeiidComponentException, ExpressionEvaluationException, TeiidProcessingException { - UpdateProcessor up = new UpdateProcessor(null, new CollectionTupleSource(tuples.iterator(), getColumns())) { - - protected void tuplePassed(List tuple) - throws BlockedException, TeiidComponentException, TeiidProcessingException { - if (rowId != null) { - tuple.add(0, rowId.getAndIncrement()); - } - insertTuple(tuple); - } - - @Override - protected void undo(List tuple) throws TeiidComponentException { - deleteTuple(tuple); - } - - }; + public TupleSource insert(TupleSource tuples, final List<ElementSymbol> variables) throws TeiidComponentException, ExpressionEvaluationException, TeiidProcessingException { + List<ElementSymbol> cols = getColumns(); + final int[] indexes = new int[cols.size()]; + boolean shouldProject = false; + for (int i = 0; i < cols.size(); i++) { + indexes[i] = variables.indexOf(cols.get(i)); + shouldProject |= (indexes[i] != i); + } + UpdateProcessor up = new InsertUpdateProcessor(tuples, rowId != null, shouldProject?indexes:null); int updateCount = up.process(); tid.setCardinality(tree.getRowCount()); return CollectionTupleSource.createUpdateCountTupleSource(updateCount); @@ -378,7 +407,7 @@ public TupleSource update(Criteria crit, final SetClauseList update) throws TeiidComponentException, ExpressionEvaluationException, TeiidProcessingException { final boolean primaryKeyChangePossible = canChangePrimaryKey(update); final TupleBrowser browser = createTupleBrower(crit, OrderBy.ASC); - UpdateProcessor up = new UpdateProcessor(crit, new TupleBrowserTupleSource(browser)) { + UpdateProcessor up = new UpdateProcessor(crit, browser) { protected TupleBuffer changeSet; protected UpdateProcessor changeSetProcessor; @@ -418,20 +447,7 @@ //changeSet contains possible updates if (primaryKeyChangePossible) { if (changeSetProcessor == null) { - changeSetProcessor = new UpdateProcessor(null, changeSet.createIndexedTupleSource(true)) { - @Override - protected void tuplePassed(List tuple) throws BlockedException, - TeiidComponentException, TeiidProcessingException { - insertTuple(tuple); - } - - @Override - protected void undo(List tuple) throws TeiidComponentException, - TeiidProcessingException { - deleteTuple(tuple); - } - - }; + changeSetProcessor = new InsertUpdateProcessor(changeSet.createIndexedTupleSource(true), false, null); } changeSetProcessor.process(); //when this returns, we're up to date } @@ -466,7 +482,7 @@ public TupleSource delete(Criteria crit) throws TeiidComponentException, ExpressionEvaluationException, TeiidProcessingException { final TupleBrowser browser = createTupleBrower(crit, OrderBy.ASC); - UpdateProcessor up = new UpdateProcessor(crit, new TupleBrowserTupleSource(browser)) { + UpdateProcessor up = new UpdateProcessor(crit, browser) { @Override protected void tuplePassed(List tuple) throws ExpressionEvaluationException, @@ -502,5 +518,5 @@ throw new AssertionError("Update failed"); //$NON-NLS-1$ } } - + } \ No newline at end of file Modified: trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/main/java/org/teiid/query/tempdata/TempTableStore.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -23,15 +23,14 @@ package org.teiid.query.tempdata; import java.util.ArrayList; +import java.util.Arrays; import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Map; import java.util.Set; -import org.teiid.api.exception.query.ExpressionEvaluationException; import org.teiid.api.exception.query.QueryProcessingException; -import org.teiid.common.buffer.BlockedException; import org.teiid.common.buffer.BufferManager; import org.teiid.common.buffer.TupleSource; import org.teiid.core.TeiidComponentException; @@ -52,7 +51,6 @@ import org.teiid.query.sql.lang.Query; import org.teiid.query.sql.lang.Update; import org.teiid.query.sql.navigator.PostOrderNavigator; -import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; @@ -142,11 +140,16 @@ final String groupKey = group.getNonCorrelationName().toUpperCase(); final TempTable table = getTempTable(groupKey, command); if (command instanceof Insert) { - List<List<Object>> tuples = getBulkRows((Insert)command, table.getColumns()); - if (tuples.isEmpty()) { - return CollectionTupleSource.createUpdateCountTupleSource(0); + Insert insert = (Insert)command; + TupleSource ts = insert.getTupleSource(); + if (ts == null) { + List<Object> values = new ArrayList<Object>(insert.getValues().size()); + for (Expression expr : (List<Expression>)insert.getValues()) { + values.add(Evaluator.evaluate(expr)); + } + ts = new CollectionTupleSource(Arrays.asList(values).iterator()); } - return table.insert(tuples); + return table.insert(ts, insert.getVariables()); } if (command instanceof Update) { final Update update = (Update)command; @@ -213,36 +216,6 @@ return groupToTupleSourceID.get(tempTableID); } - public static List<List<Object>> getBulkRows(Insert insert, List<ElementSymbol> elements) throws ExpressionEvaluationException, BlockedException, TeiidComponentException { - int bulkRowCount = 1; - if (insert.isBulk()) { - Constant c = (Constant)insert.getValues().get(0); - bulkRowCount = ((List<?>)c.getValue()).size(); - } - - List<List<Object>> tuples = new ArrayList<List<Object>>(bulkRowCount); - - for (int row = 0; row < bulkRowCount; row++) { - List<Object> currentRow = new ArrayList<Object>(insert.getValues().size()); - for (ElementSymbol symbol : elements) { - int index = insert.getVariables().indexOf(symbol); - Object value = null; - if (index != -1) { - if (insert.isBulk()) { - Constant multiValue = (Constant)insert.getValues().get(index); - value = ((List<?>)multiValue.getValue()).get(row); - } else { - Expression expr = (Expression)insert.getValues().get(index); - value = Evaluator.evaluate(expr); - } - } - currentRow.add(value); - } - tuples.add(currentRow); - } - return tuples; - } - public boolean hasTempTable(Command command){ switch (command.getType()) { case Command.TYPE_INSERT: Modified: trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java =================================================================== --- trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -36,7 +36,6 @@ import org.teiid.api.exception.query.QueryResolverException; import org.teiid.client.RequestMessage; import org.teiid.client.ResultsMessage; -import org.teiid.common.queue.FakeWorkManager; import org.teiid.dqp.internal.datamgr.ConnectorManagerRepository; import org.teiid.dqp.internal.datamgr.FakeTransactionService; import org.teiid.dqp.service.AutoGenDataService; Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -6413,12 +6413,26 @@ FakeMetadataFacade metadata = FakeMetadataFactory.example1Cached(); ProcessorPlan plan = helpPlan(sql, metadata, new String[] {"SELECT DISTINCT g_0.e1 FROM pm1.g1 AS g_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + //no txn required, since an interated insert is used + assertFalse(plan.requiresTransaction(false)); checkNodeTypes(plan, FULL_PUSHDOWN); checkNodeTypes(plan, new int[] {1}, new Class[] {ProjectIntoNode.class}); } + @Test public void testInsertQueryExpression() throws Exception { + String sql = "insert into pm1.g1 (e1) select e1 from pm1.g2"; //$NON-NLS-1$ + + FakeMetadataFacade metadata = FakeMetadataFactory.example1Cached(); + + ProcessorPlan plan = helpPlan(sql, metadata, new String[] {"SELECT g_0.e1 FROM pm1.g2 AS g_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + //requires a txn, since an non pushdown/iterated insert is used + assertTrue(plan.requiresTransaction(false)); + + checkNodeTypes(plan, new int[] {1}, new Class[] {ProjectIntoNode.class}); + } + /** * previously the subqueries were being pushed too far and then not having the appropriate correlated references */ Modified: trunk/engine/src/test/java/org/teiid/query/processor/FakeTupleSource.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/FakeTupleSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/processor/FakeTupleSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -45,8 +45,6 @@ //the first time nextTuple is called, it will throws BlockedExceptiom private boolean blockOnce; - private boolean exceptionOnClose; - public FakeTupleSource(List elements, List[] tuples) { this.elements = elements; this.tuples = tuples; @@ -113,13 +111,4 @@ this.blockOnce = true; } - public void setExceptionOnClose(boolean exceptionOnClose) { - this.exceptionOnClose = exceptionOnClose; - } - - @Override - public int available() { - return 0; - } - } Modified: trunk/engine/src/test/java/org/teiid/query/processor/TestCollectionTupleSource.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestCollectionTupleSource.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestCollectionTupleSource.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -29,8 +29,6 @@ import java.util.List; import org.junit.Test; -import org.teiid.query.processor.CollectionTupleSource; -import org.teiid.query.sql.lang.Command; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; @@ -42,11 +40,8 @@ List<SingleElementSymbol> elements = new ArrayList<SingleElementSymbol>(); elements.add(new ElementSymbol("x")); //$NON-NLS-1$ elements.add(new ElementSymbol("y")); //$NON-NLS-1$ - CollectionTupleSource nts = CollectionTupleSource.createNullTupleSource(elements); + CollectionTupleSource nts = CollectionTupleSource.createNullTupleSource(); - // Check schema - assertEquals("Didn't get expected schema", elements, nts.getSchema()); //$NON-NLS-1$ - // Walk it and get no data List tuple = nts.nextTuple(); nts.closeSource(); @@ -57,9 +52,6 @@ @Test public void testUpdateCountSource() { CollectionTupleSource nts = CollectionTupleSource.createUpdateCountTupleSource(5); - // Check schema - assertEquals("Didn't get expected schema", Command.getUpdateCommandSymbol(), nts.getSchema()); //$NON-NLS-1$ - // Walk it and get no data List tuple = nts.nextTuple(); nts.closeSource(); Modified: trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/processor/relational/TestBatchedUpdateNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -39,7 +39,6 @@ import org.teiid.query.metadata.QueryMetadataInterface; import org.teiid.query.optimizer.batch.TestBatchedUpdatePlanner; import org.teiid.query.processor.ProcessorDataManager; -import org.teiid.query.processor.relational.BatchedUpdateNode; import org.teiid.query.sql.lang.BatchedUpdateCommand; import org.teiid.query.sql.lang.Command; import org.teiid.query.sql.visitor.EvaluatableVisitor; @@ -233,7 +232,6 @@ this.numCommands = numCommands; } public void closeSource() {} - public List getSchema() {return null;} public List nextTuple() throws TeiidComponentException { if (first) { first = false; @@ -245,10 +243,6 @@ return null; } - @Override - public int available() { - return 0; - } } } Modified: trunk/engine/src/test/java/org/teiid/query/processor/relational/TestGroupingNode.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/relational/TestGroupingNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/processor/relational/TestGroupingNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -57,7 +57,7 @@ public class TestGroupingNode { - public static TupleSource createTupleSource1() { + public static FakeTupleSource createTupleSource1() { List<ElementSymbol> symbols = new ArrayList<ElementSymbol>(); symbols.add(new ElementSymbol("col1")); //$NON-NLS-1$ symbols.get(0).setType(DataTypeManager.DefaultDataClasses.INTEGER); @@ -90,7 +90,7 @@ List[] expected, ProcessorDataManager dataMgr) throws TeiidComponentException, BlockedException, TeiidProcessingException { - TupleSource dataSource = createTupleSource1(); + FakeTupleSource dataSource = createTupleSource1(); helpProcess(mgr, node, context, expected, dataSource, dataMgr); } @@ -98,7 +98,7 @@ GroupingNode node, CommandContext context, List[] expected, - TupleSource dataSource, ProcessorDataManager dataMgr) throws TeiidComponentException, + FakeTupleSource dataSource, ProcessorDataManager dataMgr) throws TeiidComponentException, BlockedException, TeiidProcessingException { RelationalNode dataNode = new FakeRelationalNode(0, dataSource, mgr.getProcessorBatchSize()); @@ -250,7 +250,7 @@ List symbols = new ArrayList(); symbols.add(bigDecimal); - TupleSource dataSource = new FakeTupleSource(symbols, data); + FakeTupleSource dataSource = new FakeTupleSource(symbols, data); helpProcess(mgr, node, context, expected, dataSource, null); } @@ -292,7 +292,7 @@ List symbols = new ArrayList(); symbols.add(col1); symbols.add(bigDecimal); - TupleSource dataSource = new FakeTupleSource(symbols, data); + FakeTupleSource dataSource = new FakeTupleSource(symbols, data); helpProcess(mgr, node, context, expected, dataSource, null); } @@ -385,7 +385,7 @@ List symbols = new ArrayList(); symbols.add(col1); symbols.add(bigDecimal); - TupleSource dataSource = new FakeTupleSource(symbols, data); + FakeTupleSource dataSource = new FakeTupleSource(symbols, data); helpProcess(mgr, node, context, expected, dataSource, null); } Modified: trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/processor/relational/TestProjectIntoNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -22,42 +22,42 @@ package org.teiid.query.processor.relational; +import static org.junit.Assert.*; + import java.util.ArrayList; import java.util.Arrays; import java.util.List; +import org.junit.Test; import org.teiid.api.exception.query.ExpressionEvaluationException; import org.teiid.common.buffer.BlockedException; import org.teiid.common.buffer.BufferManager; import org.teiid.common.buffer.TupleBatch; import org.teiid.common.buffer.TupleSource; import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidProcessingException; +import org.teiid.query.eval.Evaluator; import org.teiid.query.processor.FakeTupleSource; import org.teiid.query.processor.ProcessorDataManager; -import org.teiid.query.processor.relational.ProjectIntoNode; -import org.teiid.query.processor.relational.RelationalNode; +import org.teiid.query.processor.relational.ProjectIntoNode.Mode; import org.teiid.query.sql.lang.BatchedUpdateCommand; import org.teiid.query.sql.lang.Command; import org.teiid.query.sql.lang.Insert; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; +import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; -import org.teiid.query.tempdata.TempTableStore; import org.teiid.query.util.CommandContext; -import junit.framework.TestCase; - - - /** * @since 4.2 */ -public class TestProjectIntoNode extends TestCase { +public class TestProjectIntoNode { // Rows should be a multiple of batch size for this test to work private static final int NUM_ROWS = 1000; - private void helpTestNextBatch(int tupleBatchSize, boolean doBatching, boolean doBulkInsert, boolean exceptionOnClose) throws Exception { + private void helpTestNextBatch(int tupleBatchSize, Mode mode) throws Exception { ProjectIntoNode node = new ProjectIntoNode(2); @@ -73,14 +73,13 @@ elements.add(elementSymbol_1); elements.add(elementSymbol_2); node.setIntoElements(elements); - node.setDoBatching(doBatching); - node.setDoBulkInsert(doBulkInsert); + node.setMode(mode); node.setModelName("myModel"); //$NON-NLS-1$ CommandContext context = new CommandContext(); context.setProcessorID("processorID"); //$NON-NLS-1$ BufferManager bm = NodeTestUtil.getTestBufferManager(tupleBatchSize, tupleBatchSize); - ProcessorDataManager dataManager = new FakePDM(tupleBatchSize, exceptionOnClose); + ProcessorDataManager dataManager = new FakePDM(tupleBatchSize); child.initialize(context, bm, dataManager); node.initialize(context, bm, dataManager); @@ -106,32 +105,34 @@ assertEquals(new Integer(NUM_ROWS), columns[0]); } - public void testNextBatch() throws Exception { - helpTestNextBatch(100, true, false, false); + @Test public void testNextBatch() throws Exception { + helpTestNextBatch(100, Mode.BATCH); } - public void testNextBatch_BulkInsert() throws Exception { - helpTestNextBatch(100, false, true, false); + @Test public void testNextBatch_BulkInsert() throws Exception { + helpTestNextBatch(100, Mode.BULK); } - public void testNextBatch_NoBatching() throws Exception { - helpTestNextBatch(100, false, false, false); + @Test public void testNextBatch_NoBatching() throws Exception { + helpTestNextBatch(100, Mode.SINGLE); } - public void testNextBatch_Size20Batches() throws Exception { - helpTestNextBatch(20, true, false, false); + @Test public void testNextBatch_Size20Batches() throws Exception { + helpTestNextBatch(20, Mode.BATCH); } + @Test public void testNextBatch_Iterator() throws Exception { + helpTestNextBatch(100, Mode.ITERATOR); + } + private static final class FakePDM implements ProcessorDataManager { private int expectedBatchSize; private int callCount = 0; - private boolean exceptionOnClose; - private FakePDM(int expectedBatchSize, boolean exceptionOnClose) { + private FakePDM(int expectedBatchSize) { this.expectedBatchSize = expectedBatchSize; - this.exceptionOnClose = exceptionOnClose; } public Object lookupCodeValue(CommandContext context,String codeTableName,String returnElementName,String keyElementName,Object keyValue) throws BlockedException,TeiidComponentException {return null;} - public TupleSource registerRequest(Object processorID,Command command,String modelName,String connectorBindingId, int nodeID) throws TeiidComponentException, ExpressionEvaluationException { + public TupleSource registerRequest(Object processorID,Command command,String modelName,String connectorBindingId, int nodeID) throws TeiidComponentException, TeiidProcessingException { callCount++; int batchSize = 1; @@ -140,15 +141,23 @@ if (command instanceof Insert) { Insert insert = (Insert)command; if (insert.isBulk()) { - List batch = TempTableStore.getBulkRows(insert, insert.getVariables()); + List batch = getBulkRows(insert, insert.getVariables()); batchSize = batch.size(); assertEquals("Unexpected batch on call " + callCount, expectedBatchSize, batchSize); //$NON-NLS-1$ for (int i = 0; i < batchSize; i++) { ensureValue2((List)batch.get(i), 2, ((callCount-1) * batchSize) + i + 1); } + } else if (insert.getTupleSource() != null) { + TupleSource ts = insert.getTupleSource(); + List tuple = null; + int i = 0; + while ((tuple = ts.nextTuple()) != null) { + ensureValue2(tuple, 2, ++i); + } + batchSize = i; } else { - ensureValue((Insert)command, 2, callCount); + ensureValue(insert, 2, callCount); } } else if ( command instanceof BatchedUpdateCommand ){ BatchedUpdateCommand bu = (BatchedUpdateCommand)command; @@ -161,7 +170,6 @@ } List counts = Arrays.asList(new Object[] { new Integer(batchSize)}); FakeTupleSource fakeTupleSource = new FakeTupleSource(null, new List[] {counts}); - fakeTupleSource.setExceptionOnClose(this.exceptionOnClose); return fakeTupleSource; } @@ -201,9 +209,36 @@ ? null : Arrays.asList(new Object[] {new Integer(currentRow), Integer.toString(currentRow)}); } - @Override - public int available() { - return 0; - } } + + public static List<List<Object>> getBulkRows(Insert insert, List<ElementSymbol> elements) throws ExpressionEvaluationException, BlockedException, TeiidComponentException { + int bulkRowCount = 1; + if (insert.isBulk()) { + Constant c = (Constant)insert.getValues().get(0); + bulkRowCount = ((List<?>)c.getValue()).size(); + } + + List<List<Object>> tuples = new ArrayList<List<Object>>(bulkRowCount); + + for (int row = 0; row < bulkRowCount; row++) { + List<Object> currentRow = new ArrayList<Object>(insert.getValues().size()); + for (ElementSymbol symbol : elements) { + int index = insert.getVariables().indexOf(symbol); + Object value = null; + if (index != -1) { + if (insert.isBulk()) { + Constant multiValue = (Constant)insert.getValues().get(index); + value = ((List<?>)multiValue.getValue()).get(row); + } else { + Expression expr = (Expression)insert.getValues().get(index); + value = Evaluator.evaluate(expr); + } + } + currentRow.add(value); + } + tuples.add(currentRow); + } + return tuples; + } + } Modified: trunk/engine/src/test/java/org/teiid/query/processor/relational/TestSortNode.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/relational/TestSortNode.java 2010-07-28 21:11:54 UTC (rev 2390) +++ trunk/engine/src/test/java/org/teiid/query/processor/relational/TestSortNode.java 2010-07-29 15:18:33 UTC (rev 2391) @@ -305,7 +305,7 @@ BufferManager bm = BufferManagerFactory.getStandaloneBufferManager(); TupleBuffer tsid = bm.createTupleBuffer(Arrays.asList(es1), "test", TupleSourceType.PROCESSOR); //$NON-NLS-1$ tsid.addTuple(Arrays.asList(1)); - SortUtility su = new SortUtility(tsid.createIndexedTupleSource(), Arrays.asList(es1), Arrays.asList(Boolean.TRUE), Mode.DUP_REMOVE, bm, "test"); //$NON-NLS-1$ + SortUtility su = new SortUtility(tsid.createIndexedTupleSource(), Arrays.asList(es1), Arrays.asList(Boolean.TRUE), Mode.DUP_REMOVE, bm, "test", tsid.getSchema()); //$NON-NLS-1$ TupleBuffer out = su.sort(); TupleSource ts = out.createIndexedTupleSource(); assertEquals(Arrays.asList(1), ts.nextTuple());

14 years, 4 months

1
0
0 / 0

Teiid - Continuous Build - Build # 1232 - Unstable!

by jboss-qa-internal＠redhat.com

Teiid - Continuous Build - Build # 1232 - Unstable: Check console output at http://hudson.qa.jboss.com/hudson/job/Teiid%20-%20Continuous%20Build/1232/ to view the results.

14 years, 5 months

1
1
0 / 0

teiid SVN: r2390 - in trunk/client/src: test/java/org/teiid/jdbc and 1 other directory.

by teiid-commits＠lists.jboss.org

Author: rareddy Date: 2010-07-28 17:11:54 -0400 (Wed, 28 Jul 2010) New Revision: 2390 Modified: trunk/client/src/main/java/org/teiid/jdbc/EmbeddedProfile.java trunk/client/src/main/java/org/teiid/jdbc/SocketProfile.java trunk/client/src/test/java/org/teiid/jdbc/TestTeiidDriver.java Log: TEIID-1174: Allowing the "-" in the VDB names. Modified: trunk/client/src/main/java/org/teiid/jdbc/EmbeddedProfile.java =================================================================== --- trunk/client/src/main/java/org/teiid/jdbc/EmbeddedProfile.java 2010-07-28 19:17:38 UTC (rev 2389) +++ trunk/client/src/main/java/org/teiid/jdbc/EmbeddedProfile.java 2010-07-28 21:11:54 UTC (rev 2390) @@ -47,7 +47,7 @@ * - jdbc:teiid:BQT * - jdbc:teiid:BQT;verson=1 */ - static final String BASE_PATTERN = "jdbc:teiid:(\\w+)(;.*)?"; //$NON-NLS-1$ + static final String BASE_PATTERN = "jdbc:teiid:([\\w|-]+)(;.*)?"; //$NON-NLS-1$ private static Logger logger = Logger.getLogger("org.teiid.jdbc"); //$NON-NLS-1$ Modified: trunk/client/src/main/java/org/teiid/jdbc/SocketProfile.java =================================================================== --- trunk/client/src/main/java/org/teiid/jdbc/SocketProfile.java 2010-07-28 19:17:38 UTC (rev 2389) +++ trunk/client/src/main/java/org/teiid/jdbc/SocketProfile.java 2010-07-28 21:11:54 UTC (rev 2390) @@ -61,7 +61,7 @@ // This host/port pattern allows just a . or a - to be in the host part. static final String HOST_PORT_PATTERN = "[\\p{Alnum}\\.\\-]+:\\d+"; //$NON-NLS-1$ - static final String URL_PATTERN = "jdbc:(metamatrix|teiid):(\\w+)@mm[s]?://"+HOST_PORT_PATTERN+"(,"+HOST_PORT_PATTERN+")*(;.*)?"; //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ + static final String URL_PATTERN = "jdbc:(metamatrix|teiid):([\\w|-]+)@mm[s]?://"+HOST_PORT_PATTERN+"(,"+HOST_PORT_PATTERN+")*(;.*)?"; //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ static Pattern urlPattern = Pattern.compile(URL_PATTERN); /** Modified: trunk/client/src/test/java/org/teiid/jdbc/TestTeiidDriver.java =================================================================== --- trunk/client/src/test/java/org/teiid/jdbc/TestTeiidDriver.java 2010-07-28 19:17:38 UTC (rev 2389) +++ trunk/client/src/test/java/org/teiid/jdbc/TestTeiidDriver.java 2010-07-28 21:11:54 UTC (rev 2390) @@ -46,6 +46,8 @@ assertTrue(drv.acceptsURL("jdbc:teiid:vdb@mm://localhost:12345;user=foo;password=bar")); //$NON-NLS-1$ assertTrue(drv.acceptsURL("jdbc:teiid:vdb")); //$NON-NLS-1$ assertFalse(drv.acceptsURL("jdbc:teiid:vdb@/foo/blah/deploy.properties")); //$NON-NLS-1$ + assertTrue(drv.acceptsURL("jdbc:teiid:8294601c-9fe9-4244-9499-4a012c5e1476_vdb")); //$NON-NLS-1$ + assertTrue(drv.acceptsURL("jdbc:teiid:8294601c-9fe9-4244-9499-4a012c5e1476_vdb@mm://localhost:12345")); //$NON-NLS-1$ }

14 years, 5 months

1
0
0 / 0

teiid SVN: r2389 - in trunk/engine/src: test/java/org/teiid/dqp/internal/process and 1 other directory.

by teiid-commits＠lists.jboss.org

Author: rareddy Date: 2010-07-28 15:17:38 -0400 (Wed, 28 Jul 2010) New Revision: 2389 Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestPreparedStatement.java Log: TEIID-1104 Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java 2010-07-28 19:04:04 UTC (rev 2388) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/SessionAwareCache.java 2010-07-28 19:17:38 UTC (rev 2389) @@ -61,7 +61,7 @@ SessionAwareCache (int maxSize, final CacheFactory cacheFactory, final Cache.Type type){ if(maxSize < 0){ - this.maxSize = DEFAULT_MAX_SIZE_TOTAL; + maxSize = DEFAULT_MAX_SIZE_TOTAL; } this.maxSize = maxSize; this.localCache = new DefaultCache<CacheID, T>("local", maxSize); //$NON-NLS-1$ @@ -133,7 +133,7 @@ private String sessionId; private String originalSessionId; private List<Serializable> parameters; - private String userName; + //private String userName; boolean cachable = true; CacheID(DQPWorkContext context, ParseInfo pi, String sql){ @@ -141,7 +141,7 @@ this.vdbInfo = new VDBKey(context.getVdbName(), context.getVdbVersion()); this.pi = pi; this.originalSessionId = context.getSessionId(); - this.userName = context.getUserName(); + //this.userName = context.getUserName(); } private void setSessionId(String sessionId) { @@ -163,7 +163,7 @@ } public void setUserName(String name) { - this.userName = name; + //this.userName = name; } public boolean equals(Object obj){ @@ -175,13 +175,13 @@ } CacheID that = (CacheID)obj; return this.pi.equals(that.pi) && this.vdbInfo.equals(that.vdbInfo) && this.sql.equals(that.sql) - && EquivalenceUtil.areEqual(this.userName, that.userName) + //&& EquivalenceUtil.areEqual(this.userName, that.userName) && EquivalenceUtil.areEqual(this.sessionId, that.sessionId) && EquivalenceUtil.areEqual(this.parameters, that.parameters); } public int hashCode() { - return HashCodeUtil.hashCode(0, vdbInfo, sql, pi, sessionId, parameters); + return HashCodeUtil.hashCode(0, vdbInfo, sql, pi, sessionId, parameters); } @Override Modified: trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestPreparedStatement.java =================================================================== --- trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestPreparedStatement.java 2010-07-28 19:04:04 UTC (rev 2388) +++ trunk/engine/src/test/java/org/teiid/dqp/internal/process/TestPreparedStatement.java 2010-07-28 19:17:38 UTC (rev 2389) @@ -159,7 +159,8 @@ @Test public void testSessionSpecificFunction() throws Exception { // Create query - String preparedSql = "SELECT user(), e2, pm1.g1.e3 as a, e4 as b FROM pm1.g1 WHERE e2=?"; //$NON-NLS-1$ + System.setProperty("foo", "foo"); //$NON-NLS-1$ //$NON-NLS-2$ + String preparedSql = "SELECT env('foo'), e2, pm1.g1.e3 as a, e4 as b FROM pm1.g1 WHERE e2=?"; //$NON-NLS-1$ // Create expected results List[] expected = new List[] {

14 years, 5 months

1
0
0 / 0

teiid SVN: r2388 - trunk/jboss-integration/src/main/java/org/teiid/adminapi/jboss.

by teiid-commits＠lists.jboss.org

Author: rareddy Date: 2010-07-28 15:04:04 -0400 (Wed, 28 Jul 2010) New Revision: 2388 Modified: trunk/jboss-integration/src/main/java/org/teiid/adminapi/jboss/Admin.java Log: TEIID-1169 Modified: trunk/jboss-integration/src/main/java/org/teiid/adminapi/jboss/Admin.java =================================================================== --- trunk/jboss-integration/src/main/java/org/teiid/adminapi/jboss/Admin.java 2010-07-28 18:27:57 UTC (rev 2387) +++ trunk/jboss-integration/src/main/java/org/teiid/adminapi/jboss/Admin.java 2010-07-28 19:04:04 UTC (rev 2388) @@ -70,7 +70,6 @@ import org.teiid.adminapi.impl.VDBMetaData; import org.teiid.adminapi.impl.WorkerPoolStatisticsMetadata; import org.teiid.jboss.IntegrationPlugin; -import org.teiid.jboss.deployers.RuntimeEngineDeployer; public class Admin extends TeiidAdmin { private static final String CONNECTOR_PREFIX = "connector-"; //$NON-NLS-1$ @@ -78,7 +77,7 @@ private static final long serialVersionUID = 7081309086056911304L; private static ComponentType VDBTYPE = new ComponentType("teiid", "vdb");//$NON-NLS-1$ //$NON-NLS-2$ private static ComponentType DQPTYPE = new ComponentType("teiid", "dqp");//$NON-NLS-1$ //$NON-NLS-2$ - private static String DQPNAME = RuntimeEngineDeployer.class.getName(); + private static String DQPNAME = "RuntimeEngineDeployer"; //$NON-NLS-1$ private static ComponentType TRANSLATOR_TYPE = new ComponentType("teiid", "translator");//$NON-NLS-1$ //$NON-NLS-2$ private static final String[] DS_TYPES = {"XA", "NoTx", "LocalTx"}; //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$

14 years, 5 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-commits July 2010