SSU Development thread (4.0 to 5.0) [DEVELOPMENT HALTED DUE TIME REQUIREMENTS!]

[Urwumpe]

Which is what? As I wrote, I've been using this method for years and have never encountered any certification issues. It does this once, the first time you connect to the server and then never again.

So I fail to see the so called "hassle".

Well, I don't put my private key password in plaintext on my machine...

 

[GLS]

Heads-up from SF:
SVN timeouts are to be expected today intermittently while our DevOps team reboots server hosts in our cluster.

---------- Post added 04-24-18 at 07:38 PM ---------- Previous post was 04-23-18 at 11:04 PM ----------

I sure as **** hope this is not permanent:
Empty Repository
It looks like this Subversion repository doesn't have any files in it. Let's commit your project code now.

 

[DaveS]

Heads-up from SF:
SVN timeouts are to be expected today intermittently while our DevOps team reboots server hosts in our cluster.

---------- Post added 04-24-18 at 07:38 PM ---------- Previous post was 04-23-18 at 11:04 PM ----------

I sure as **** hope this is not permanent:
Empty Repository
It looks like this Subversion repository doesn't have any files in it. Let's commit your project code now.

What if it is? What's forward plan? No more revision history. Should we just take decision right here and now to leave SF? It seems to me the one SF had going for it was the SVN revision history and that's now gone with the digital winds.

 

[GLS]

... and it's back.
I'll try the merge once again.

---------- Post added at 08:09 PM ---------- Previous post was at 08:07 PM ----------

What if it is? What's forward plan? No more revision history. Should we just take decision right here and now to leave SF? It seems to me the one SF had going for it was the SVN revision history and that's now gone with the digital winds.

They have backups, so only recent changes would be lost... hopefully... :uhh:
IMO leaving SF is already decided. Where to... that's the million dolar question. Right now I'd like to square things before changing homes, but it should be this summer.

---------- Post added at 08:18 PM ---------- Previous post was at 08:09 PM ----------

First run went OK.
Second run failed in the same place.... :facepalm:

 

[GLS]

So confident am I that SF will fix things by tomorrow evening, that I'd like to start planning for the migration process now, and maybe do it in the weekend. We could also evaluate now a possible change in the version control protocol.
The way I see it we have 6 options:
1) SVN @ SF
2) SVN @ OH
3) SVN @ elsewhere
4) git @ OH
5) git @ GitHub
6) git @ SF
7) git @ elsewhere

(obviously, option 1... is no longer an option)

In all honesty, I'm not (yet) fully confortable with git, but that's probably all in my head. That and the fact that (usually) there's something to gain from leaving your comfort zone, I vote for a switch to git*.
As for where our stuff goes, anywhere should be better that SF. One thing to consider though, are the tickets. If we are to leave SF, I say we close shop in there, so the ticket data (the open ones at least) must go somewhere else.


*) provided we keep the project history and we (at least initially) follow a centralized scheme.

 

[Face]

The way I see it we have 6 options:
1) SVN @ SF
2) SVN @ OH
3) SVN @ elsewhere
4) git @ OH
5) git @ GitHub
6) git @ SF
7) git @ elsewhere

With 2, you'd have to think about ticket migration. Either you stick with SF, ask the O-F crew to give you a project in the vBulletin tracker, or you migrate to a 3rd party. With 3, the elsewhere might have a proper ticket system.

For 4, I don't know if there even is an OH git server. On 6, you'd just exchange the VCS on SF, sticking with the crappy service that might also affect their git offerings. 5 would be an option that is mainstream, having all things in one place. For 7, elsewhere could be the number 2 in the DVCS hub world: Bitbucket of Atlassian.

For the version control alone, given git's DVCS nature, the choice of blessed repo hosting is not so important. You can switch and/or duplicate the hosting in no time.
Tickets are a more risky decision. First, migration tools from one side to another are not so common. If you are lucky enough to find one, they are often of beta quality with no support.
Second, the ticket workflow is normally centralized, so it is important where your ivory tower stands. Nothing more annoying than fixing a bug and being ready to commit, only to find your ticket system being down, rendering you unable to get that dreaded ticket number you want to include in your commit. And even if you have dutifully noted it down when starting to code, you can't close the ticket or comment on it during the ticket system downtime, while the repo(s) already have the bug-fix committed, pushed, and perhaps even CI-built and deployed.

I'd suggest 5. Go with Github. It might have its sharp edges, but it is mainstream, and nothing beats the broad front of integrations, clients, help-databases and pre-built infrastructures specialized on Github. SVN and SF might still have many years to go, but they already fall behind in terms of peer pressure.

 

[GLS]

For the tickets, if we could come up with some rules to handle them here in a thread (or threads), that would work for me. :shrug:

For 4, I don't know if there even is an OH git server.

I don't either. I just feel blessed this place exists and wished I could help support it.

On 6, you'd just exchange the VCS on SF, sticking with the crappy service that might also affect their git offerings.

Oh, it does... :facepalm:

 

[DaveS]

I vote 2) as the problem is not with SVN but rather Source Forge. SVN is perfectly adequate for our needs and we have bigger fish to catch than the protocol we're using. All our downtime in the last few years have been due to Source Forge server issues which are completely unrelated to the SVN protocol. SF doesn't equal SVN and SVN doesn't equal SF.

Other much bigger projects than ours are perfectly fine using SVN, but they're hosted elsewhere than SF which just proves that SF is the issue here, not SVN so let's place the blame where it properly belongs, on SF's deteriorating reliability and service in the last few years.

I have personally dealt with Xyon (primary OH/O-F admin) lately and I can vouch for the first class support and service we'll get here. And besides, if the O-F/OH servers take a nosedive, it will affect alot more than our capability of doing SVN stuff, like our ability to communicate (well we do have the back up of e-mail I suppose).

So to summarize everything, I say SVN with vBulletin project for the tickets which is very similar to what SF is providing.

 

[jedidia]

For 7, elsewhere could be the number 2 in the DVCS hub world: Bitbucket of Atlassian.

Wouldn't that also be the case for #3? I think Atlassian supports svn as well.

EDIT: Never mind, I'm stupid. Atlassian tools offer support to cooperate with svn repositories, but bitbucket doesn't host any.

 

[Face]

I have personally dealt with Xyon (primary OH/O-F admin) lately and I can vouch for the first class support and service we'll get here.

Well, I remember that the Orbiter beta SVN repo once lost its history due to an SVN migration. I don't know if it was deliberate or not, or if it was necessary to start on a clean slate. I'd suggest to contact Xyon on the technicalities of a possible migration to OHM before deciding on this.

---------- Post added at 18:09 ---------- Previous post was at 17:25 ----------

I think the above mentioned migration happened during O-F's hosting changes: https://www.orbiter-forum.com/showthread.php?p=444802

 

[kuddel]

Hi there,
I've followed the latest discussion a little (not in every detail) and just have to add this, I think:

Whatever provider/host you will choose (or stay at SF), there is no excuse not to make backups!
Every single storage can fail. Accidentally, intentionally or for whatever other reason.

AND, as backing up the repository is very easy, there is one excuse less

svnrdump dump svn://svn.code.sf.net/p/shuttleultra/code/trunk > SSU.svndump
...that's all.
It takes a while depending on overall throughput, but after that you have a full backup of the repository[1], that can be very helpful in case of a failure.

Needless to say, but backups only really make sense if done regularly.

So, when there are 4 main developers and every one does a kind of backup (more sophisticated incremental backups might be better, though) the repository / history is pretty save.

Just my 2cents,
Kuddel

[1] in this example only /trunk/ is remotely dumped, so it's actually not a full full backup

 

[DaveS]

Hi there,
I've followed the latest discussion a little (not in every detail) and just have to add this, I think:

Whatever provider/host you will choose (or stay at SF), there is no excuse not to make backups!
Every single storage can fail. Accidentally, intentionally or for whatever other reason.

AND, as backing up the repository is very easy, there is one excuse less

svnrdump dump svn://svn.code.sf.net/p/shuttleultra/code/trunk > SSU.svndump
...that's all.
It takes a while depending on overall throughput, but after that you have a full backup of the repository[1], that can be very helpful in case of a failure.

Needless to say, but backups only really make sense if done regularly.

So, when there are 4 main developers and every one does a kind of backup (more sophisticated incremental backups might be better, though) the repository / history is pretty save.

Just my 2cents,
Kuddel

[1] in this example only /trunk/ is remotely dumped, so it's actually not a full full backup

I just finished a full dump of the SSU SVN repository, not just the trunk. 4.11 GB total, around 5 minutes to dump it.

 

[GLS]

Well, looks like they have given up... and so have I.

Hello,

Thank you for the update. We apologize for https:// not working properly for you, please use svn+ssh, that should work more reliabily.

Sincerely,

SourceForge Support

I have no problems staying with SVN.
If we move to here, do we have a "proper" ticket system? If so, and if we can "clone" our svn history, then let's do it. :thumbup:

BTW: I'm also doing the backup.

 

[DaveS]

Well, looks like they have given up... and so have I.


I have no problems staying with SVN.
If we move to here, do we have a "proper" ticket system? If so, and if we can "clone" our svn history, then let's do it. :thumbup:

BTW: I'm also doing the backup.

Seems like the actual migration is very easy: https://www.petefreitag.com/item/665.cfm. And as far as the ticket system is concerned, I think it would be very easy to set up an additional vBulletin project here on O-F. Then we could simply just copy and paste the details of the tickets on SF to the new entries here on O-F. The numbers would differ of course but I would consider that a minor thing.

 

[Face]

I just finished a full dump of the SSU SVN repository, not just the trunk. 4.11 GB total, around 5 minutes to dump it.

Huh? Is there so much meta-data (properties?) in it? Or is this some uncompressed format? My copy is 1GB total, and last I've checked it contains all revisions.

---------- Post added at 07:49 ---------- Previous post was at 07:32 ----------

Forget it.. found my answer. It looks like the various SVN dump methods (svnadmin, svnrdump) use different formats. svnadmin without the deltas option produces a big uncompressed dump. With the option it gets close to the "normal" size. svnrdump OTOH seems to always compress in order to reduce network traffic. What method did you use to produce the 4GB dump?

 

[DaveS]

Huh? Is there so much meta-data (properties?) in it? Or is this some uncompressed format? My copy is 1GB total, and last I've checked it contains all revisions.

---------- Post added at 07:49 ---------- Previous post was at 07:32 ----------

Forget it.. found my answer. It looks like the various SVN dump methods (svnadmin, svnrdump) use different formats. svnadmin without the deltas option produces a big uncompressed dump. With the option it gets close to the "normal" size. svnrdump OTOH seems to always compress in order to reduce network traffic. What method did you use to produce the 4GB dump?

I used svnrdump.

 

[kuddel]

svnrdump (svnremotedump) has the one big advantage, that it works without having direct access to the repository.
So everyone that has "read access" can create dumps.

On the repository-server itself other methods might be more advantageous.
Incremental backups can be done with both methods.
I have a .bat script somewhere around doing that. I'll post it when I find it (don't expect that to happen too soon - real life, you know... )

 

[GLS]

So, do we bring our stuff to OF and keep SVN? If so how to we get that in motion?

 

[DaveS]

So, do we bring our stuff to OF and keep SVN? If so how to we get that in motion?

I've asked Xyon in a PM to chime in here on this subject. Haven't heard anything back yet.

 

[Xyon]

Yeah, sorry about that, my availability this week has been dreadful.

Reading back down the last few posts, it's clear there are a few issues with your current setup. This ties into a few things that are (slowly) happening behind the scenes with OHM in general.

The current plan runs thus:

1. Develop a RESTful API to serve as the core of the OrbitHangar repository site
2. Develop a new website frontend to leverage this core API
3. Implement an authentication "hook" to provide some form of single-sign on between the forum and the API core
4. Link the API to additional services such as GitLab, IRC notification, etc

Some of the items on this list have had some work put to them, but none of them are yet production-ready. In particular, we have a copy of GitLab running on my kit for testing purposes, within which the current code for the RESTful API is hosted (using git, of course). This will ultimately form an attached service as part of the final list item once the other pieces are in place, but naturally things only proceed as free time allows us to make progress on them.

It would be possible (and quite easy) to afford this project space within the aforementioned GitLab instance, but that would naturally mean the project migrating to use git instead of SVN. There is not currently an SVN server set up for the purposes of OHM as such, but I do run a subversion server for the D3D9 client (again, on my kit) which, equally, I can provide SSU space in if retaining SVN is desirable. This may over time become integrated with OHM itself as the GitLab instance, though this is a more convoluted process.

So, in short, you can have your cake and eat it just fine if you want, though the current state is much less mature than I'd like.