[rules-dev] Split-up: the gory details (for future reference)
Geoffrey De Smet
ge0ffrey.spam at gmail.com
Tue Mar 1 06:56:02 EST 2011
For future reference, I am copy-pasting the methodology I used during
the split-up in our dev mailing list archive.
Just ignore it.
However, do take a look at the blog about the split-up, which has some
interesting numbers:
http://blog.athico.com/2011/03/splitting-up-into-smaller-git.html
--
With kind regards,
Geoffrey De Smet
split-up
========
Gather all in one directory
---------------------------
Comment out the modules with: <!-- Already split-off to separate repo -->
Run a clean build: mvn clean install -DskipTests
Commit with message: JBRULES-2869: Split-off preparation: move
everything for a separate repository together into a single directory
Do split
--------
git clone git at github.com:droolsjbpm/droolsjbpm.git droolsjbpm-untouched
cd droolsjbpm-untouched/
git fetch
git branch
git pull --rebase
cd ..
cp -R droolsjbpm-untouched <repoName>-uncloned
cd <repoName>-uncloned
git remote rm origin
git branch -a
git tag -l
git tag -d 3.0-GA
git tag -d 3.0-GA at 4643
git tag -d 3.0.1-GA
git tag -d 3.0.2-GA
git tag -d 3.0.3-GA
git tag -d 3.0.5-GA
git tag -d 3.0.6-GA
git tag -d 4.0.0.13773GA
git tag -d 4.0.1.14754GA
git tag -d 4.0.2-SOA_4.2.CP01
git tag -d 4.0.2.15666.GA
git tag -d 4.0.2.15830.SOA-4-2.CP01
git tag -d 4.0.3.15993.GA
git tag -d 4.0.4.17825GA
git tag -d 4.0.5.19064GA
git tag -d 4.0.6.19326.GA
git tag -d 4.0.7.19894.GA
git tag -d 5.0.0.26115.FINAL
git tag -d 5.0.1.26597.FINAL
git tag -d 5.1.0.34406.FINAL
git tag -d 5.1.1.34858.FINAL
git tag -d 5.2.0.M1
git tag -d DROOLS_SOA_4_2_18483_GA
git tag -d DROOLS_SOA_4_2_19342_CP1
git tag -d DROOLS_SOA_4_2_19937_CP2
git tag -d DROOLS_SOA_4_2_CR3_18258
git tag -d before_merge_factsconstraint
git tag -l
Depending on the split-off:
Canonical: git filter-branch --index-filter 'git ls-tree --name-only
--full-tree $GIT_COMMIT | grep -v "^\(\(\.git\)\|...\)$" | xargs -I {}
git rm --cached -r {}' --tag-name-filter cat --prune-empty -- --all
For planner
-----------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(drools\-docs\)\|\(drools-planner\)\|\(drools-solver\)\)$"
| xargs -I {} git rm --cached -r {}' --tag-name-filter cat --prune-empty
-- --all
The list: drools-docs drools-planner drools-solver
For tools
---------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(droolsjbpm\-tools\)\|\(drools\-ant\)\|\(drools\-eclipse\)\|\(drools\-ide\)\|\(drools\-ide\-feature\)\|\(drools\-ide\-test\)\|\(drools\-eclipse3\.2\)\|\(drools\-eclipse3\.3\)\)$"
| xargs -I {} git rm --cached -r {}' --tag-name-filter cat --prune-empty
-- --all
The list: droolsjbpm-tools drools-ant drools-eclipse drools-ide
drools-ide-feature drools-ide-test drools-eclipse3.2 drools-eclipse3.3
For guvnor
----------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(drools\-docs\)\|\(drools\-examples\)\|\(guvnor\)\|\(drools\-guvnor\)\|\(drools\-jbrms\)\|\(drools\-atom\)\|\(drools\-factconstraint\)\|\(drools\-ide\-common\)\|\(drools\-repository\)\|\(drools\-repository\-parent\)\)$"
| xargs -I {} git rm --cached -r {}' --tag-name-filter cat --prune-empty
-- --all
The list: drools-docs drools-examples guvnor drools-guvnor drools-jbrms
drools-atom drools-factconstraint drools-ide-common drools-repository
drools-repository-parent
Also remove the GWT generated binaries from the history:
git filter-branch --index-filter 'git ls-tree --name-only --full-tree -r
$GIT_COMMIT | grep --regex
"^.*/src/main/webapp/.*\(\(\.gwt\.rpc\)\|\(\.cache\.html\)\|\(\.cache\.js\)\|\(\.cache\.xml\)\)$"
| xargs -I {} git rm --cached {}' --tag-name-filter cat --prune-empty --
--all
The list: src/main/webapp/org.drools.guvnor.Guvnor/**/*.gwt.rpc
src/main/webapp/org.drools.guvnor.Guvnor/**/*.cache.html
src/main/webapp/org.drools.guvnor.Guvnor/**/*.cache.js
src/main/webapp/org.drools.guvnor.Guvnor/**/*.cache.xml
For integration
---------------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(drools\-docs\)\|\(drools\-grid\)\|\(drools\-container\)\|\(drools\-spring\)\|\(drools\-seam\)\|\(drools\-camel\)\|\(drools\-server\)\|\(drools\-pipeline\)\|\(drools\-rhq\-plugin\)\|\(drools\-simulator\)\)$"
| xargs -I {} git rm --cached -r {}' --tag-name-filter cat --prune-empty
-- --all
The list: drools-docs drools-grid drools-container drools-spring
drools-seam drools-camel drools-server drools-pipeline drools-rhq-plugin
drools-simulator
For drools
----------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep
"^\(\(blog\)\|\(lib\)\|\(m2_repo\)\|\(repository\)\|\(drools\-flow\-compiler\)\|\(drools\-flow\-core\)\|\(drools\-flow\-persistence\-jpa\)\|\(drools\-process\)\|\(drools-planner\)\|\(drools-solver\)\|\(droolsjbpm\-tools\)\|\(drools\-ant\)\|\(drools\-eclipse\)\|\(drools\-ide\)\|\(drools\-ide\-feature\)\|\(drools\-ide\-test\)\|\(drools\-eclipse3\.2\)\|\(drools\-eclipse3\.3\)\|\(guvnor\)\|\(drools\-guvnor\)\|\(drools\-jbrms\)\|\(drools\-atom\)\|\(drools\-factconstraint\)\|\(drools\-ide\-common\)\|\(drools\-repository\)\|\(drools\-repository\-parent\)\|\(drools\-grid\)\|\(drools\-container\)\|\(drools\-spring\)\|\(drools\-seam\)\|\(drools\-camel\)\|\(drools\-server\)\|\(drools\-pipeline\)\|\(drools\-rhq\-plugin\)\|\(drools\-simulator\)\|\(drools\-api\)\|\(.*\.xml\)\|\(.*\.txt\)\|\(.*\.TXT\)\|\(.*\.env\)\|\(.*\.sh\)\|\(.*\.properties\)\(\.gitignore\)\|\(src\)\|\(install\)\|\(osgi\-bundles\)\)$"
| xargs -I {} git rm --cached -r {}' --tag-name-filter cat --prune-empty
-- --all
The list is NOT: blog lib m2_repo repository drools-flow-compiler
drools-flow-core drools-flow-persistence-jpa drools-process + rest of
other migrations also not except drools-examples and drools-docs
For droolsjbpm-knowledge
------------------------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(drools\-docs\)\|\(drools\-api\)\)$" | xargs -I {} git
rm --cached -r {}' --tag-name-filter cat --prune-empty -- --all
The list: drools-docs drools-api
For droolsjbpm-build-bootstrap
----------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(.*\.xml\)\|\(.*\.txt\)\|\(.*\.TXT\)\|\(.*\.env\)\|\(.*\.sh\)\|\(.*\.properties\)\(\.gitignore\)\)$"
| xargs -I {} git rm --cached -r {}' --tag-name-filter cat --prune-empty
-- --all
The list: pom.xml README.txt LICENSE-ASL-2.0.txt eclipse-cleanup.xml
eclipse-code-style.xml
For droolsjbpm-build-distribution
---------------------------------
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v
"^\(\(\.git\)\|\(src\)\|\(install\)\|\(osgi\-bundles\)\)$" | xargs -I {}
git rm --cached -r {}' --tag-name-filter cat --prune-empty -- --all
The list: src install osgi-bundles
Then continue with:
-------------------
cd ..
git clone
file:///home/gdesmet/projects/jboss/split-up-migration/<repoName>-uncloned/
<repoName>
cd <repoName>
git remote rm origin
git remote add origin git at github.com:droolsjbpm/<repoName>.git
git push --all
After the migration
===================
Remove from monolithic-droolsjbpm
---------------------------------
Delete the directories split-off in the droolsjbpm repository.
Comment out their <module> reference in the parent pom: <!-- Already
split-off to separate repo -->
Run:
mvn clean install -DskipTests
Commit with message: JBRULES-2869: Split-off of module GGG into separate
git repository
Clean up in new repository
--------------------------
Remove non-needed docs and examples dirs.
Commit with message: JBRULES-2869: Split-off: remove directories from
docs and examples that do not belong in the repository
Add license file
----------------
Copy the LICENSE-ASL-2.0.txt file
Commit with message: JBRULES-2869: Split-off requires license file per repo
Hook in multiproject
--------------------
Hook in the new multiproject parent pom.
mvn clean install -DskipTests
Commit with message: JBRULES-2869: Hook in the new multiproject as
parent after split-up
dos2unix
--------
Run command:
find . -type f -regex ".*\.java$" -and -not -path \*/\.git/\* -exec
dos2unix {} \;
find . -type f -regex ".*\.xml$" -and -not -path \*/\.git/\* -exec
dos2unix {} \;
find . -type f -not -path \*/\.git/\* -exec dos2unix {} \;
Review files that you don't want to dos2unix
Commit with message: JBRULES-2855 Dos2unix all source files (except .bat)
Copyright javadoc
-----------------
Replace in all *.java files regex case sensitive:
/\*\*\n \* Copyright
with
/*\n * Copyright
Search in all files regex case sensitive:
/\*\*\n \* Copyright
Commit with message: JBRULES-2577 License headers in java files are not
valid javadocs and should not start with /**
Remove all \t suffixes
----------------------
Replace in all *.java files regex case sensitive:
([;\{\}])[\t ]+$
with
$1
Commit with message: JBRULES-2855 Remove all trailing whitespace in java
files
Tabs
----
Replace in all *.java/drl files regex case sensitive:
\t
with 4 spaces
Or replace multiple times in all *.java/drl files regex case sensitive:
^( *)\t
with (without quotes) "$1 "
Replace in all *.xml files regex case sensitive:
\t
with 2 spaces
Search in all files regex:
\t
Commit with message: JBRULES-2855 Do not use tabs (\t) for indentation,
use spaces instead: 4 for java files, 2 for xml, 4 for drl
Strip author
------------
Replace in all *.java files regex case sensitive:
^ *\* + at author .*\n
with nothing.
Delete empty javadocs.
Replace in all *.java files regex case sensitive:
^ */\*\*\n( *\* *\n)* *\*/\n
with nothing
Delete trailing empty javadoc lines:
Replace in all *.java files regex case sensitive:
^( *\* *\n)+( *)\*/
with
$2*/
Search in all files:
author
Commit with message: JBRULES-2895 Strip all author tags from java files
- For motivation see https://issues.jboss.org/browse/JBRULES-2895
Rename to new directory structure
---------------------------------
Do renames
Commit with message: JBRULES-2869: Rename to new directory structure
after split-up
Add repository jboss public in main pom.xml:
<repositories>
<!-- Bootstrap repository to locate the parent pom when the parent pom
has not been build locally. -->
<repository>
<id>jboss-public-repository-group</id>
<name>JBoss Public Repository Group</name>
<url>http://repository.jboss.org/nexus/content/groups/public/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
</repository>
</repositories>
Commit with message: JBRULES-2869: Include bootstrap repository
Configure hudson job
--------------------
TODO for planner and tools
+ mutliproject pom per repo
# parent repository in it
# correct parent pom path
Failed alternatives
===================
Probably not needed:
git checkout --track remotes/origin/3.0.x
git checkout --track remotes/origin/4.0.x
git checkout --track remotes/origin/4.1.x
git checkout --track remotes/origin/5.0.x
git checkout --track remotes/origin/5.1.x
git checkout --track remotes/origin/5.2.0.M1.x
git checkout --track remotes/origin/K200
git checkout --track remotes/origin/lr_unlinking_20101116
git checkout --track remotes/origin/persistence_refactor
git checkout --track remotes/origin/rete_using_static_methods_aug2009
git checkout master
Alt 1:
cp -R droolsjbpm-untouched drools-planner-uncloned
cd drools-planner-uncloned
git remote rm origin
git remote -v
git branch -D K200
git branch -D lr_unlinking_20101116
git branch -d persistence_refactor
git branch -D rete_using_static_methods_aug2009
git branch -d 3.0.x
git filter-branch --subdirectory-filter drools-solver --tag-name-filter
cat 4.0.x
git filter-branch --subdirectory-filter drools-solver --tag-name-filter
cat -f 4.1.x
git filter-branch --subdirectory-filter drools-solver --tag-name-filter
cat -f 5.0.x
git filter-branch --subdirectory-filter drools-planner --tag-name-filter
cat -f 5.1.x
git filter-branch --subdirectory-filter drools-planner --tag-name-filter
cat -f 5.2.0.M1.x
git filter-branch --subdirectory-filter drools-planner --tag-name-filter
cat -f master
cd ..
git clone
file:///home/gdesmet/projects/jboss/split-up-migration/drools-planner-uncloned
drools-planner
cd drools-planner
git checkout --track remotes/origin/4.0.x
git checkout --track remotes/origin/4.1.x
git checkout --track remotes/origin/5.0.x
git checkout --track remotes/origin/5.1.x
git checkout --track remotes/origin/5.2.0.M1.x
git checkout master
git remote rm origin
git remote add origin git at github.com:droolsjbpm/drools-planner.git
git push --all
Alt 2:
git clone --no-hardlinks
file:///home/gdesmet/projects/jboss/split-up-migration/droolsjbpm-untouched
drools-planner-uncloned
cd drools-planner-uncloned
git remote rm origin
git filter-branch --subdirectory-filter drools-planner -- --all
git reset --hard
git gc --prune
cd ..
git clone
file:///home/gdesmet/projects/jboss/split-up-migration/drools-planner-uncloned
drools-planner
cd drools-planner
git remote rm origin
git remote add origin git at github.com:droolsjbpm/drools-planner.git
git push --all
Alt 3:
git filter-branch --tree-filter 'rm -Rf lib' -- --all
for i in $(ls) ; do
echo $i
done
git filter-branch --tree-filter "find . -not -regex '\.' -and -not
-regex '\./git.*' -and -not -regex '\./drools-planner.*' -and -not
-regex '\./drools-solver.*' -and -not -regex
'\./drools-docs/drools-docs-planner.*' -and -not -regex
'\./drools-docs/drools-docs-solver.*' -delete" --tag-name-filter cat
--prune-empty -- --all
Alt 4:
git remote rm origin
git filter-branch --subdirectory-filter drools-planner --tag-name-filter
cat --prune-empty -- --all
=> anything before containing "drools-planner" stays as-is so it stays 1GB
Alt 5:
git remote rm origin
git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch
lib drools-api drools-core drools-compiler' --tag-name-filter cat
--prune-empty -- --all
Alt 6:
git remote rm origin
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v "^\(\.git\)\|\(drools\-ant\)\|\(drools\-eclipse\)"
| xargs git rm -r --cached --ignore-unmatch {}' --prune-empty master
-- --all
Alt 7: => WINNER
git remote rm origin
git branch -a
git branch -D 3.0.x
git branch -D 4.0.x
git branch -D 4.1.x
git branch -D 5.0.x
git branch -D 5.1.x
git branch -D 5.2.0.M1.x
git branch -D K200
git branch -D lr_unlinking_20101116
git branch -D persistence_refactor
git branch -D rete_using_static_methods_aug2009
git branch -a
git tag -l
git tag -d 3.0-GA
git tag -d 3.0-GA at 4643
git tag -d 3.0.1-GA
git tag -d 3.0.2-GA
git tag -d 3.0.3-GA
git tag -d 3.0.5-GA
git tag -d 3.0.6-GA
git tag -d 4.0.0.13773GA
git tag -d 4.0.1.14754GA
git tag -d 4.0.2-SOA_4.2.CP01
git tag -d 4.0.2.15666.GA
git tag -d 4.0.2.15830.SOA-4-2.CP01
git tag -d 4.0.3.15993.GA
git tag -d 4.0.4.17825GA
git tag -d 4.0.5.19064GA
git tag -d 4.0.6.19326.GA
git tag -d 4.0.7.19894.GA
git tag -d 5.0.0.26115.FINAL
git tag -d 5.0.1.26597.FINAL
git tag -d 5.1.0.34406.FINAL
git tag -d 5.1.1.34858.FINAL
git tag -d 5.2.0.M1
git tag -d DROOLS_SOA_4_2_18483_GA
git tag -d DROOLS_SOA_4_2_19342_CP1
git tag -d DROOLS_SOA_4_2_19937_CP2
git tag -d DROOLS_SOA_4_2_CR3_18258
git tag -d before_merge_factsconstraint
git tag -l
git filter-branch --subdirectory-filter drools-planner --prune-empty master
# extra guvnor step?
cd ..
git clone
file:///home/gdesmet/projects/jboss/split-up-migration/drools-planner-uncloned/
drools-planner
cd drools-planner
git remote rm origin
git remote add origin git at github.com:droolsjbpm/drools-planner.git
git push --all
Alt 8:
git filter-branch --index-filter 'git ls-tree --name-only --full-tree
$GIT_COMMIT | grep -v "^\(\.git\)\|\(drools\-ant\)\|\(drools\-eclipse\)"
| xargs -0 -I {} git rm --cached --ignore-unmatch -r {}' --prune-empty
master
Hickups
-------
Redo planner migration by starting from an earlier tree:
git reset --hard 73add05c3fd74b3fdf4456055d486b7a7547c28b
More information about the rules-dev
mailing list