Project Common Orbiter Installation

[Face]

Release 1.0.1 is out.

Download

Wiki

In addition to the above mentioned inclusion of the "Add" feature, this release fully supports updates by means of selfhosted COI selections.
This means, that you can do the following after initial installation of the COI client:

* Start the application
* Browse the local site to your applications installation directory (e.g. "C:\Programs\COI Client")
* Answer all questions regarding creation of COI repository with Yes.
* Enter the COIClient's COI address in the remote box: http://bitbucket.org/face/coiclientcoi
* Click refresh on remote pane
* Select the one entry in remote (Common Object Installation Client 1.0.1)
* Click the download button
* The repository will be downloaded to the installation directory. Vista users may need to give the COIClient admin privilegs for this to do...
* Select the single "addon" in the local pane and click "Process". The application will open a temporary batch file to update itself and exit. After the update is complete, the application will restart.

Whenever a new version comes out now, I'll update the COIClient's COI, you can download it and activate the new version immediately without further installation procedures. OFC the MSI will get updated, too.

ATTENTION: If you want to test the potential of COI for use as Orbiter addon manager, I can offer to host a temporary test environment for you to pull from. OFC this cannot be persistent (or public) as it would infringe copyrights. Everyone interested can contact me via email, PM or IRC...

regards,
Face

 

[tblaxland]

Some comments on the new version:

1. I connected to the remote repo you provided on bitbucket and successfully downloaded the COI Client addon that was in there. I also added an addon to the local COI successfully. :speakcool:

2. If I clone http://bitbucket.org/face/coiclientcoi to a local directory and point COI Client at it for the remote repo, I get: ""C:\coiclientcoi" is no COI - database repository is missing". I found that I could get it to work if I made a repo in "C:\OrbiterCOI", then created "C:\OrbiterCOI\.hg\coi" and copied the contents of "C:\coiclientcoi" into it. Is it necessary (or desirable) for local and http repos to have a different structure like this?

3. When you "Process Selections" after selecting an addon, the COI Client window closes and opens a command prompt window (I guess it is running some hg commands) and the COI Client window reopens when it is finished. Can you make the COI Client window persistent, or at least re-open in the same spot (by default Windows moves it around so that the windows cascade)? I note that deselecting an addon then clicking "Process Selections" does not bring up a command prompt window, it just shows info in the status bar.

4. I think the "Add" button should be relabelled "Add/Edit" to more accurately reflect its function.

5. At the moment, if I want to edit an addon, I have to open the Add dialog and search through the changesets for the addon I want to edit. It would be better if I could just click on it in the "Local" pane of the main client window, then click "Add/Edit" and that changeset is already presented.

Good work mate.

 

[Face]

2. If I clone http://bitbucket.org/face/coiclientcoi to a local directory and point COI Client at it for the remote repo, I get: ""C:\coiclientcoi" is no COI - database repository is missing". I found that I could get it to work if I made a repo in "C:\OrbiterCOI", then created "C:\OrbiterCOI\.hg\coi" and copied the contents of "C:\coiclientcoi" into it. Is it necessary (or desirable) for local and http repos to have a different structure like this?

Yes, the system makes a difference between file-system and non-file-system. If a file-system location is selected, it assumes the location to be the actual repository and the coi-data to reside in .hg/coi/. If a remote location is used, it assumes the location to be the coi-data and the actual repository is pointed to by means of the .url file in the coi-data. This way, I can create simple clones locally and still have the flexibility to host COIs almost anywhere. An example would be to have a COI hosted by means of the "hg serve" feature. It can server the coi-data repo at e.g. port 8000 and the actual repo at 8001 while having a .url file pointing to port 8001.
It seems a little cumbersome to do this manually, but a serving and cloning feature is planned anyway. The goal is to shield as much VCS-things as possible from the user.

3. When you "Process Selections" after selecting an addon, the COI Client window closes and opens a command prompt window (I guess it is running some hg commands) and the COI Client window reopens when it is finished. Can you make the COI Client window persistent, or at least re-open in the same spot (by default Windows moves it around so that the windows cascade)? I note that deselecting an addon then clicking "Process Selections" does not bring up a command prompt window, it just shows info in the status bar.

This is only the case with COIs of the COIClient itself. It is necessary to do this in order to update the application itself.
With all other COIs this shouldn't be the case.

4. I think the "Add" button should be relabelled "Add/Edit" to more accurately reflect its function.

Indeed. Will be done in the next version.

5. At the moment, if I want to edit an addon, I have to open the Add dialog and search through the changesets for the addon I want to edit. It would be better if I could just click on it in the "Local" pane of the main client window, then click "Add/Edit" and that changeset is already presented.

This is already on the TODO.

Thanks for trying it out...

cheers,
Face

 

[tblaxland]

Thanks for trying it out...

You're most welcome. Thanks for the explanations - as usual, you have thought through all my concerns ahead of time

Some more comments:

1. I forgot to mention this one yesterday. When I first started using COI Client I found the Commit and Add buttons in the Add/Edit dialog confusing with regard to their intended purpose. Perhaps this could be resolved by providing them with more verbose labels such as "Commit New Files as Addon" and "Add Addon to COI". Maybe you could also move the Commit button to the left hand side of the dialog rather than having it grouped with the Add/Edit & Cancel buttons, especially since Commit and Add/Edit are quite separate operations.

2. I created two local COIs (OrbiterCOI1 and OrbiterCOI2) and put an addon in each using COI Client (TestAddon1 and TestAddon2). I then created a third local COI (COIClientTest) and pointed the remote side at OrbiterCOI1, downloaded and installed TestAddon1 successfully. I then pointed the remote side at OrbiterCOI2 and got an error "abort: repository is unrelated". While this is not unexpected (hence why I was testing for it) and I understand what is going on from a DVCS point of view, it would be great if this could be somehow resolved. Envisage a scenario where you and I are both hosting COIs that we have created independently and are containing a different subset of addons. The user would need separate local COIs to be able to access those remote COIs, which seems to defeat the purpose of having COIs. Is it possible to give COI Client the ability to merge unrelated remote COIs into the local COI? Like this: http://mercurial.selenic.com/wiki/MergingUnrelatedRepositories

 

[Face]

1. I forgot to mention this one yesterday. When I first started using COI Client I found the Commit and Add buttons in the Add/Edit dialog confusing with regard to their intended purpose. Perhaps this could be resolved by providing them with more verbose labels such as "Commit New Files as Addon" and "Add Addon to COI". Maybe you could also move the Commit button to the left hand side of the dialog rather than having it grouped with the Add/Edit & Cancel buttons, especially since Commit and Add/Edit are quite separate operations.

Hm. You have a point there. I just think the suggested labels would be to long for a button.
Maybe moving and adding the labels as tooltips will do the trick?

2. I created two local COIs (OrbiterCOI1 and OrbiterCOI2) and put an addon in each using COI Client (TestAddon1 and TestAddon2). I then created a third local COI (COIClientTest) and pointed the remote side at OrbiterCOI1, downloaded and installed TestAddon1 successfully. I then pointed the remote side at OrbiterCOI2 and got an error "abort: repository is unrelated". While this is not unexpected (hence why I was testing for it) and I understand what is going on from a DVCS point of view, it would be great if this could be somehow resolved. Envisage a scenario where you and I are both hosting COIs that we have created independently and are containing a different subset of addons. The user would need separate local COIs to be able to access those remote COIs, which seems to defeat the purpose of having COIs. Is it possible to give COI Client the ability to merge unrelated remote COIs into the local COI? Like this: http://mercurial.selenic.com/wiki/MergingUnrelatedRepositories

Understood. This is easily dealed with by means of the force option. I'll put it in.
Normally, unrelated repos should not be pulled mutually, but in this case it makes sense to have two roots and merge them together.

The original idea to use COI for Orbiter, however, was a common installation starting at the very base: Orbiter's base package. This way, the root would be the same for all used COIs.

This is a bit of a problem, since it would be natural for users to start with COI by downloading Orbiter base and commiting it as first "addon". If everyone does this independently, it would produce many different base installations, since the time, the user and perhaps the commit message would be different.
My idea here is to simply skip the downloading of the ZIP. Why not downloading by means of COI to start with. If we put up a repository hosting the "one and only" Orbiter base changeset, it would ease up things.
Imagine the casual Orbiter newbie... instead of going to www.orbitersim.com, finding and downloading a ZIP package and extracting it with directory structure preserved, all he/she does is entering www.orbitersim.com/coi in the remote pane and select the Orbiter "addon". After downloading he/she can select it in the local pane and process. It is about the same procedure complexity-wise.
Hence the term Common Orbiter Installation.

Of course there is another route to go: skip the base. Don't start the COI with the base installation as the root. Just start with addons...

cheers,
Face

 

[tblaxland]

Hm. You have a point there. I just think the suggested labels would be to long for a button.
Maybe moving and adding the labels as tooltips will do the trick?

That sounds like a good solution to me.

Imagine the casual Orbiter newbie... instead of going to www.orbitersim.com, finding and downloading a ZIP package and extracting it with directory structure preserved, all he/she does is entering www.orbitersim.com/coi in the remote pane and select the Orbiter "addon". After downloading he/she can select it in the local pane and process. It is about the same procedure complexity-wise.
Hence the term Common Orbiter Installation.

I find it hard to disagree with any of that.

---------- Post added 03-09-09 at 10:32 ---------- Previous post was 02-09-09 at 17:31 ----------

Understood. This is easily dealed with by means of the force option. I'll put it in.
Normally, unrelated repos should not be pulled mutually, but in this case it makes sense to have two roots and merge them together.

Would it be possible to show the contents of the remote COI before doing the merge, and only doing the merge after the first download from that repo is requested? It would also be good if a warning dialog pops up before merging the two roots. Those two things combined should help prevent someone merging the roots of two completely unrelated COIs, eg, you would want to avoid merging a COI for Orbiter with a COI for JoeBlogsWizzBangProject. EDIT: Another feature that might help in that regard would be that if a user selects a different local COI, the remote COI changes to the last remote COI used with that particular local COI, or empty if the last remote COI used cannot be determined (that's quite a tortured sentence, I hope it makes sense )

EDIT2: One more feature request (you're going to be sick of me before to long :dry: Can you please make COI Client remember its window size settings when closing/opening?

EDIT3: What is the purpose of the Lock button in the Add/Edit dialog?

 

[Face]

Would it be possible to show the contents of the remote COI before doing the merge, and only doing the merge after the first download from that repo is requested? It would also be good if a warning dialog pops up before merging the two roots. Those two things combined should help prevent someone merging the roots of two completely unrelated COIs, eg, you would want to avoid merging a COI for Orbiter with a COI for JoeBlogsWizzBangProject.

Showing the contents of the remote COI-data is in fact merging it with the local COI-data. There is no problem with this IMHO. Actually downloading an addon from an unrelated repository is the thing to worry about. I can give the --force option to the COI-data retrival and put a message box up for the downloading procedure whenever the pull would say "unrelated"...

EDIT: Another feature that might help in that regard would be that if a user selects a different local COI, the remote COI changes to the last remote COI used with that particular local COI, or empty if the last remote COI used cannot be determined (that's quite a tortured sentence, I hope it makes sense )

Surely helpful, but not so easy to implement. I'll put it on the TODO for later versions, though.

EDIT2: One more feature request (you're going to be sick of me before to long :dry: Can you please make COI Client remember its window size settings when closing/opening?

No big deal. Wilco. (And your input is welcome!)

EDIT3: What is the purpose of the Lock button in the Add/Edit dialog?

Normally the user/name/version-data is updated whenever you select a revision which is marked as addon. If you want to move or replace addons, you can lock the data in order to keep it while selecting another revision. A typical use-case is applying a small fix to an addon without changing the version. You'd then just move the addon description to another revision. A replace would be a move to a revision already marked as addon...

cheers,
Face

 

[tblaxland]

I can give the --force option to the COI-data retrival and put a message box up for the downloading procedure whenever the pull would say "unrelated"...

Cheers.

Surely helpful, but not so easy to implement. I'll put it on the TODO for later versions, though.

Fair enough.

Normally the user/name/version-data is updated whenever you select a revision which is marked as addon. If you want to move or replace addons, you can lock the data in order to keep it while selecting another revision. A typical use-case is applying a small fix to an addon without changing the version. You'd then just move the addon description to another revision. A replace would be a move to a revision already marked as addon...

Thanks for that. I actually used that function this weekend when I was doing some transplanting/stripping in a repo and it was very helpful.

Another issue that became apparent in my attempts to break COI Client was that it could be easy to inadvertently commit files that are automatically created to an unrelated addon. Scenario:
1) I extract, commit and add IMFD and proceed to use it. In doing so, it creates an unmanaged log file.
2) I update to revision 0, and extract and commit Attitude MFD. If I am not careful, IMFD's log file will be committed with Attitude MFD since, to the casual eye, it appears no different to any of the other unknown files listed in hgtk commit.

For a log file, that is probably not the worst sin known to man, but it does happen with other automatically created files. For now, I added this .hgignore to my repo:

relre:^.hgignore$
relre:^keymap.cfg$
relre:^Device.dat$
relre:^Scenarios\/\(Current state\).scn
relre:^Orbiter.cfg$
relre:^Orbiter.log$

I'm undecided on whether or not the .hgignore should be included in the repo. Any thoughts on that?

I've attached a screenie of the repo I built for testing, including the hgtk log.

 

[Face]

Another issue that became apparent in my attempts to break COI Client was that it could be easy to inadvertently commit files that are automatically created to an unrelated addon. Scenario:
1) I extract, commit and add IMFD and proceed to use it. In doing so, it creates an unmanaged log file.
2) I update to revision 0, and extract and commit Attitude MFD. If I am not careful, IMFD's log file will be committed with Attitude MFD since, to the casual eye, it appears no different to any of the other unknown files listed in hgtk commit.

I usually handle these problems by means of the "purge" command. It will clean out the whole repo from auto-generated files. I thought about adding a "Cleaning" section to the menu where the user can purge the working dir, remove automerges and clean-up unused coi-data entries in order to keep the COI clean.

For a log file, that is probably not the worst sin known to man, but it does happen with other automatically created files. For now, I added this .hgignore to my repo:

relre:^.hgignore$
relre:^keymap.cfg$
relre:^Device.dat$
relre:^Scenarios\/\(Current state\).scn
relre:^Orbiter.cfg$
relre:^Orbiter.log$

I'm undecided on whether or not the .hgignore should be included in the repo. Any thoughts on that?

I always include the .hgignore in the repo. In fact it is often the very first commit.

I've attached a screenie of the repo I built for testing, including the hgtk log.

Looks cool. We should try to merge our COIs to see what happens.

Your COI shows to me that a grouping feature is needed. Maybe additional coi-data "Group" and "Class". A order-by feature would be nice, too...

cheers,
Face

---------- Post added at 22:49 ---------- Previous post was at 13:59 ----------

1.0.2 is up, tblaxland's suggestions are in. Only the repository was updated, should work fine via COI itself...

 

[tblaxland]

1.0.2 is up, tblaxland's suggestions are in. Only the repository was updated, should work fine via COI itself...

Indeed it does. Thanks for that, I'll come back with comments later.

 

[tblaxland]

I usually handle these problems by means of the "purge" command. It will clean out the whole repo from auto-generated files.

Thanks for the tip.

I thought about adding a "Cleaning" section to the menu where the user can purge the working dir, remove automerges and clean-up unused coi-data entries in order to keep the COI clean.

That sounds like a good idea, I managed to generate a good number of automerges when I was playing with this yesterday. I assume you would only strip automerges that are heads, and you would need to repeat until there are no more automerge heads in the case where you have automerges of other automerges.

Regarding stripping, yesterday I was doing some transplanting and stripping in the repo and I found "strip" to take a very long time to complete (approx 10 min). It seemed to be independent of the size of the changesets being stripped (typically approx 1MB), and dependent on the size of the repo (approx 300MB, excluding the working directory). Some experimenting suggested that the "qpop" command (which "strip" uses AFAIK) was responsible. It was very CPU intensive (90%+, used for the compression of the changesets?). Is there a better way?

I always include the .hgignore in the repo. In fact it is often the very first commit.

I've added it now, I can see no good reason not to.

Looks cool. We should try to merge our COIs to see what happens.

:rofl:
Practically, I don't know how to do that as I don't have any ready means of making a 300MB+ repo available to you.

Your COI shows to me that a grouping feature is needed. Maybe additional coi-data "Group" and "Class". A order-by feature would be nice, too...

"Class" as in: vessel, textures, MFD, etc? "Group" as in STS, DG-IV, XR-series, etc? That sounds cool. Order-by, or sort-by, would be great too.

I have some additional comments:

1. In the add/edit dialog, could you display the commit message instead of the changeset hash?

2. The window size/position stays put, thanks. Is it too much trouble to add the "list control" (or whatever you call it) settings to the list of things being stored (column width, column order - I prefer addon/version/author myself, sort-by)?

3. I have a terminology nitpick (see attached picture). "Author" is better, IMHO.

 

[Face]

That sounds like a good idea, I managed to generate a good number of automerges when I was playing with this yesterday. I assume you would only strip automerges that are heads, and you would need to repeat until there are no more automerge heads in the case where you have automerges of other automerges.

Well, I'm not entirely sure about the exact procedure, but something along the lines of Git's garbage collection. I.e. stripping away all changesets not reachable via tags and/or addons. This would include all automerges.

Regarding automerges:

Missing Features

• Feature selection complex merge algorithm (reuse of past conflict resolutions).

This means that there will be less automerges in the end, as past merges will be reused more often with new addon combinations.

Regarding stripping, yesterday I was doing some transplanting and stripping in the repo and I found "strip" to take a very long time to complete (approx 10 min). It seemed to be independent of the size of the changesets being stripped (typically approx 1MB), and dependent on the size of the repo (approx 300MB, excluding the working directory). Some experimenting suggested that the "qpop" command (which "strip" uses AFAIK) was responsible. It was very CPU intensive (90%+, used for the compression of the changesets?). Is there a better way?

Yeah, I've experienced it myself with TortoiseHg's stripping. I think it is due to the default backup-creation. Try the commandline with "hg strip -n <rev>", this will not create a backup-bundle. Should speed up the process somewhat...

Practically, I don't know how to do that as I don't have any ready means of making a 300MB+ repo available to you.

Well, I could open up my private COI repo temporarly. Just give me a time-window, preferable at the weekend...

1. In the add/edit dialog, could you display the commit message instead of the changeset hash?

This is somewhat trickier to do, as I have to decode it first from the DAG. Having the commit-message outputted will certainly increase the decode time. I'll put it on the list for later, though...

2. The window size/position stays put, thanks. Is it too much trouble to add the "list control" (or whatever you call it) settings to the list of things being stored (column width, column order - I prefer addon/version/author myself, sort-by)?

This would be cool, too, but I prefer to put this on the list for later, since it would go nicely along with the changes regarding order/sort-by.

3. I have a terminology nitpick (see attached picture). "Author" is better, IMHO.

Good catch, missed that. Will fix...

Next is remove- and upload-feature. We'll then have the core-features complete. The roadmap will then be:

• Add serve and clone feature
• Enhance addon list with grouping, sorting and preview features
• Refine inner mechanisms (automerge picking, download granularity, DAG decoding)
• Add dependency reordering feature
• Beta release 1.9
• Bug fixes
• Release 2.0 on OH
• Lobbying for OH COI

cheers,
Face

 

[tblaxland]

Yeah, I've experienced it myself with TortoiseHg's stripping. I think it is due to the default backup-creation. Try the commandline with "hg strip -n <rev>", this will not create a backup-bundle. Should speed up the process somewhat...

I tried that, but qpop still creates a temp file (slightly smaller than the repo) in the strip-backup folder. I don't think the file write is the problem anyway since the time taken is much longer than I would expect to write a 300MB file (I can download a 300MB file from the internet in roughly the same amount of time). The CPU seems to be the choke point. Perhaps it would be quicker to do a "clone to rev" and then apply patches from the original repo.

Well, I could open up my private COI repo temporarly. Just give me a time-window, preferable at the weekend...

You have a PM.

Next is remove- and upload-feature. We'll then have the core-features complete. The roadmap will then be:

• Add serve and clone feature
• Enhance addon list with grouping, sorting and preview features
• Refine inner mechanisms (automerge picking, download granularity, DAG decoding)
• Add dependency reordering feature
• Beta release 1.9
• Bug fixes
• Release 2.0 on OH

:speakcool:

• Lobbying for OH COI

:lol:

 

[Face]

I tried that, but qpop still creates a temp file (slightly smaller than the repo) in the strip-backup folder. I don't think the file write is the problem anyway since the time taken is much longer than I would expect to write a 300MB file (I can download a 300MB file from the internet in roughly the same amount of time). The CPU seems to be the choke point.

Hm. Maybe you had to strip away changesets before (time-wise) large commits. It would then qpop away the big changesets on top of the repo, delete the small old commits and qpush the big ones again.
Still puzzling, 'cause I've done that multiple times with my own repo without observing long delays in the ranks you mentioned.

Did you activate the --git option for MQ? I have the following in my mercurial.ini:

[ui]
ssh = "C:\Program Files\TortoiseHg\TortoisePlink.exe" -ssh -2
username = Face
merge = kdiff3

[merge-tools]
kdiff3.priority=-1
kdiff3.args=-L1 base --L2 local --L3 other $base $local $other -o $output
kdiff3.regkey=Software\KDiff3
kdiff3.regappend=\kdiff3.exe
kdiff3.fixeol=True
kdiff3.gui=True
beyondcompare3.priority=-2
beyondcompare3.args=$local $other $base $output /ro /lefttitle=local /centertitle=base /righttitle=other /automerge /reviewconflicts /solo
beyondcompare3.regkey=Software\Scooter Software\Beyond Compare 3
beyondcompare3.regname=ExePath
beyondcompare3.gui=True
diffmerge.priority=-7
diffmerge.args=--nosplash --merge --title1=base --title2=local --title3=other $base $local $other
diffmerge.checkchanged=True
diffmerge.gui=True
p4merge.priority=-8
p4merge.args=$base $local $other $output
p4merge.regkey=Software\Perforce\Environment
p4merge.regname=P4INSTROOT
p4merge.regappend=\p4merge.exe
p4merge.gui=True
tortoisemerge.priority=-9
tortoisemerge.args=/base:$output /mine:$local /theirs:$other /merged:$output
tortoisemerge.regkey=Software\TortoiseSVN
tortoisemerge.gui=True
winmergeu.regkey=Software\Thingamahoochie\WinMerge\
winmergeu.regname=Executable
winmergeu.priority=-10
winmergeu.args=/e /ub /dl other /dr local $other $local $output
winmergeu.fixeol=True
winmergeu.gui=True

[tortoisehg]
commit = internal
vdiff = vdiff
graphlimit = 500
authorcolor = True
copyhash = True
longsummary = True
vdiffnowin = True

[extensions]
acl = 
bookmarks =
bugzilla = 
children = 
churn =
color = !
convert = 
extdiff =
fetch = 
gpg = 
graphlog = 
hgk = 
hgcia =
highlight = !
inotify = !
interhg = 
keyword = 
mq = 
notify = 
parentrevspec = 
patchbomb =
purge = 
pager =
rebase =
record = 
transplant = 
win32text =
win32mbcs =

; Extra extensions
makewritable =
timestamp =
fold =
forest =

; List extensions added manually in Contrib
shelve = c:\program files\tortoisehg\Contrib\hgshelve.py
qup = c:\program files\tortoisehg\Contrib\qup.py

[extdiff]
cmd.vdiff = C:\Program Files\WinMerge\WinMergeU.exe
opts.vdiff = /e /x /ul /ur /maximize

 [hooks]
update.timestamp = python:hgext.timestamp.hook

; The git extended diff format will properly store binary files,
; file permission changes, and rename information that the normal
; patch format cannot cover.  However it is also not 100% compatible
; with tools which expect normal patches. so enable these at your
; own risk.
;
[diff]
git = true
;nodates = false

[alias]
qstatus = qseries -v
slog = glog -l 16 --style compact
shelved = unshelve -i | more

[bookmarks]
track.current = true

Perhaps it would be quicker to do a "clone to rev" and then apply patches from the original repo.

Interesting you mention this. I've done a little test at work to check HG's hardlink-facility. If you clone to rev, you won't get hardlinks, so the cloning takes longer and uses up space. If you clone fully, the cloning is almost instant and takes up almost no space, because hardlinks are used. If you then strip away some changesets, the space used is more, but still less then using cloning to rev. And it is somewhat faster.
I think it depends on the position of the revs, though...

regards,
Face

---------- Post added at 08:39 ---------- Previous post was at 08:37 ----------

I'll happily test next release for you, Face. Just gimme a shout.

SHOUT! :lol:

 

[tblaxland]

Hm. Maybe you had to strip away changesets before (time-wise) large commits. It would then qpop away the big changesets on top of the repo, delete the small old commits and qpush the big ones again.

That would explain the large temp file, as that was indeed the case. I don't understand why the more recent commits need to be qpop'd/qush'd if they are in an unrelated branch. It that certainly seems to be what is happening though.

During my experimenting today, I actually rebuilt a copy of the repo by transplanting the branches over one at a time (there were not many branches). I then patched up the .addons file with the correct hashes since the layout of the repo was slightly different (I moved the .hgignore down to rev 1, for example). It all worked nicely, if a little tedious.

Did you activate the --git option for MQ? I have the following in my mercurial.ini:

No, I didn't know I should. I'll give it a try.

Interesting you mention this. I've done a little test at work to check HG's hardlink-facility. If you clone to rev, you won't get hardlinks, so the cloning takes longer and uses up space. If you clone fully, the cloning is almost instant and takes up almost no space, because hardlinks are used. If you then strip away some changesets, the space used is more, but still less then using cloning to rev. And it is somewhat faster.
I think it depends on the position of the revs, though...

Interesting, I didn't know that local clones used hard links. You learn something everyday...

 

[Face]

Update on performance tests

DISCLAIMER: All the tests were run on one machine client-side, but on 3 different machines server-side. In addition, I used the client machine's connection for browsing in the meantime (although in reduced quantities and for text only), so don't take them as scientifically sound.

I've run some tests to check the download speed for addons. For this I've created a COI consisting only of the Orbiter base install (O2K6P1). If I download the package as ZIP, I get this timing on my server machine:

snoopie:/***/repos# time wget [URL]http://download.orbit.m6.net/orbiter060929_base.zip[/URL]
--09:21:49--  [URL]http://download.orbit.m6.net/orbiter060929_base.zip[/URL]
           => `orbiter060929_base.zip'
Auflösen des Hostnamen »download.orbit.m6.net«.... 74.86.130.70
Verbindungsaufbau zu download.orbit.m6.net[74.86.130.70]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 63,627,079 [application/x-zip-compressed]

100%[====================================================================================================>] 63,627,079   356.16K/s    ETA 00:00

09:24:47 (349.37 KB/s) - »orbiter060929_base.zip« gespeichert [63627079/63627079]


real    2m58.333s
user    0m0.850s
sys     0m2.510s

As you can see, the download of approx. 63MB took roughly 3 minutes. That's about 350kB/s, matching my home connection's max. download speed.

If I unpack this ZIP to a directory, I get this timing:

snoopie:/***/repos/coirepo# time unzip orbiter060929_base.zip
..
..
..
real    0m21.397s
user    0m4.320s
sys     0m1.230s

So it took about 21.5 seconds to unpack it. In summary, this is approx. 3.33 minutes to get Orbiter in a running state via ZIP procedure with my connection.

Now I uploaded a COI consisting solely of this download to hosts providing the repository in static and dynamic form via static-http protocol and WSGI protocol, respectively.

Then I cloned the COI first from the static site:

snoopie:/***/repos# time hg clone -Ur0 static-http://***/coirepo
destination directory: coirepo
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 935 changes to 935 files

real    5m1.145s
user    0m54.160s
sys     0m7.790s

This took a little over 5 mins to download.

Then I used the dynamic site:

snoopie:/***/repos# time hg clone -Ur0 [URL]http://***/coirepo[/URL]
real URL is [URL]http://*:8000/**/coirepo[/URL]
destination directory: coirepo
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 935 changes to 935 files

real    3m38.148s
user    0m45.320s
sys     0m7.400s

This was a bit faster with roughly 3.66 mins.

Then I updated the repo to the appropriate revision aka unpack Orbiter:

snoopie:/***/repos/coirepo# time hg up -v
..
..
..
935 files updated, 0 files merged, 0 files removed, 0 files unresolved

real    0m19.890s
user    0m8.870s
sys     0m2.330s

About the same as with unzip...

Getting Orbiter to run from scratch is:

• 3.33 mins (3.0m down + 20s extract) for conventional method
• 5.33 mins (5.0m down + 20s extract) for slowest protocol COI
• 3.66 mins (3.3m down + 20s extract) for (supposedly) fastest protocol COI

The size of the ZIP is 63,627,079 Bytes, the COI repo is 64,487,425 Bytes:

snoopie:/***/repos/coirepo# du -bs .hg
64487425        .hg

So to summarize:
Getting Orbiter from COI rather then from ZIP seems to be no big difference ON MY MACHINE. It would be interesting to know how other testers' mileage varies here (thanks to tblaxland for already testing this on his end).

So to everyone interested: please contact me and tell me where to send you the appropriate download links so you can get into the game...

regards,
Face

 

[wehaveaproblem]

Did I hear you yelling, Face?

I'm busy as buggery atm, so just been chilling in my free time, but I shall check this for def over the weekend mate.

 

[Face]

Progress...

Version 1.0.5 is out. Download from the usual place as MSI setup or just use COI itself to upgrade.

This version's highlight is the remove dialog. You can now get rid of those addons again.

If you select an addon in the local pane, your Remove button will be enabled (red circle):


You can click the button to bring up the remove dialog (or simply press DEL - ENTER will bring the Add/Edit dialog):


In this dialog, you have the revision selection combo and the log button on top, a treeview of derived addons in the middle and checkboxes for single/derived remove- and strip-actions at the bottom.
The combo and log button works like in the Add/Edit dialog. Your selected addon's revision will be set as default.
The treeview shows you the selected addon together with all derived addons (as apposed to the view of the local pane - showing all dependencies).
If you check the box for removal of derived addons, all addons in the treeview will be removed from your COI on clicking the Remove button. If not, only the top addon will be removed. Just removing will only affect the information about a certain revision, not the revision itself.
If you check the box for strip, the removal of derived addons is implicit and you'll get a warning message:

Stripping will delete the revision(s) from your COI and you'd have to restore them from remote locations to get them back. Just be aware, that stripping can take a long time...

If you have only removed information from a revision, you can still select it in the combobox. In this case, the top "addon" displayed in the treeview will be only a placeholder. In addition, the removal of derived addons is implicit, so the checkbox is disabled and checked:


Wether you remove or strip the addons, they will not show up in your local pane anymore, but linked remote COIs will show them as downloadable again:


regards,
Face

 

[tblaxland]

just use COI itself to upgrade.

Done, worked nicely.

This version's highlight is the remove dialog. You can now get rid of those addons again.

Works :speakcool:. I was debating with myself whether or not it would be worth having a strip-backup option, but decided against it since it is so easy to add things in again from the remote repo.

Good work mate.

 

[Face]

Progress... 1.1.0 is out

This marks "feature complete" state, so all initially planned features are in. Upload of addons and selection of remote branch have been added, so the cycle of push and pull is complete now.

The remote pane contains a drop-down-button to select a branch on the remote side. By entering arbitrary text in the textbox of the drop-down-list, you can create new branches. By right-clicking branch-entries and clicking the remove context-entry, you can delete them again.
These "branches" create partitions of the remote location's database. You can use them to e.g. separate users on big repositories or keep different addon-domains appart (historical, current, near-future, star-trek, etc.).

The "tip"-branch is the default mainline.

The upload-buttons open a simple upload-dialog. There you can select the branch for the newly uploaded addon(s). In addition, whenever you select "tip" on http-/ssh-repositories, the publish checkbox is enabled. Selecting this box will cause a global tag be created for the uploaded database change, moving the "tip" marker up to the new database state. Without this, you will still see the remote repository without the addon.

Most of the time, you want to publish the addon right away on upload. But with bigger platforms, the non-publish strategie might come handy. We'll see..

I will postpone further development on this now and focus a bit more on OMP the next month. Bug reports and suggestions are still welcome, though...

regards,
Face