Difference between revisions of "Git"

From LinuxMIPS
Jump to: navigation, search
(The queue branch now lives in its own repository.)
(Which git protocol to use: - document https)
(18 intermediate revisions by 8 users not shown)
Line 14: Line 14:
  
 
== Accessing GIT repositories ==
 
== Accessing GIT repositories ==
The GIT repositories can be accessed by <tt>git://ftp.linux-mips.org/pub/scm</tt>.  The http and rsync protocols are also supported but not recommended.  Here are two example commands using the <tt>linux.git</tt> repository; you can substitute the name of another repository (see further below) for <tt>linux.git</tt> in the examples.
+
The GIT repositories can be accessed by <tt>git://git.linux-mips.org/pub/scm</tt>.  The http and rsync protocols are also supported but not recommended.  Here are two example commands using the <tt>linux.git</tt> repository; you can substitute the name of another repository (see further below) for <tt>linux.git</tt> in the examples.
  
 
=== Cloning a repository ===
 
=== Cloning a repository ===
  
     git clone git://ftp.linux-mips.org/pub/scm/linux.git linux.git
+
     git clone git://git.linux-mips.org/pub/scm/ralf/linux
 +
 
 +
This will create a new directory under the current working directory called linux.
 +
To use a different name specify it on the command line as an additional argument and it will become the target.
 +
Here is an alternate form of the command which creates a local directory called linux.git instead of linux.
 +
 
 +
    git clone git://git.linux-mips.org/pub/scm/ralf/linux linux.git
  
 
=== Updating a repository ===
 
=== Updating a repository ===
 +
When the repository was cloned the upstream URL was saved into the file <tt>.git/remotes/origin</tt>.
 +
This is the default location for git to pull updates.
 +
If you wish to modify the default URL you may edit that file directly.
 +
 
From the top directory of your local repository (that is, the directory which contains the <tt>.git</tt> subdirectory) run the following command:
 
From the top directory of your local repository (that is, the directory which contains the <tt>.git</tt> subdirectory) run the following command:
  
     git pull git://ftp.linux-mips.org/pub/scm/linux.git
+
    git pull
 +
 
 +
This will pull updates from the repository and merge them into your local repository.
 +
 
 +
It is also possible to specify an alternate URL by specifying it on the command line.
 +
 
 +
     git pull git://git.linux-mips.org/pub/scm/ralf/linux
  
This will pull updates from the repository and merge them into your local repositoryAnd because this is actually a whole lot to type, git will remember the original URL you have used when cloning in the file <tt>.git/remotes/origin</tt>, so you can just use <tt>git pull</tt> and it'll do (almost) the same thing as if you specified the URL.
+
Typing in URLs can be tediousYou may create shortcuts to repeatedly used URLs by saving them in a remotes file.
 +
Alternate URLs may be saved in the <tt>.git/remotes/REMOTENAME</tt> file where the <tt>REMOTENAME</tt> string is the name that you wish to call that source location.  Let's say that that you wanted to clone this long URL
 +
<tt>git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git</tt> but find that a lot to type repeatedly.
 +
If that is specified in a remotes file <tt>linus</tt> then the following command would pull from it. <tt>git pull linus</tt>
  
 
=== Which git protocol to use ===
 
=== Which git protocol to use ===
Line 31: Line 50:
 
* git Git's own protocol which tries to heavily optimize the amount of bandwidth used and thus is generally very efficient for updates.  An issue with the git protocol is its use of TCP port 9418 which [[Wikipedia:Paranoia|paranoid]] firewall admins may have blocked.
 
* git Git's own protocol which tries to heavily optimize the amount of bandwidth used and thus is generally very efficient for updates.  An issue with the git protocol is its use of TCP port 9418 which [[Wikipedia:Paranoia|paranoid]] firewall admins may have blocked.
 
* http Rather inefficient usage of bandwith and CPU but since http is generally enabled in firewalls it exists for those poor souls suffering from [[Wikipedia:Fascist|fascist]] firewall admins.
 
* http Rather inefficient usage of bandwith and CPU but since http is generally enabled in firewalls it exists for those poor souls suffering from [[Wikipedia:Fascist|fascist]] firewall admins.
* rsync The oldest git protocol, deprecated and supposed to eventually go away.  Suffers from a low probability race condition.  Its advantage is the lowest CPU usage on the server side.  Not recommended for pulling or fetching.  Heck, it really should be considered the last alternative.
+
* https More secure than http but otherwise pretty much the same thing.  linux-mips.org uses SSL certificates from cacert.org and CACERT's root certificate is not included with most operating systems.  A notable exception is Debian however.  So if attempts to access a linux-mips.org repository via https fail with an error message about an invalid SSL certificate, you can:
 +
** use one of the other protocols listed in this section
 +
** disable the verification of the SSL certificate.  On a UNIX or Linux type of system you do this by setting the GIT_SSL_NO_VERIFY variable, for example: <code>GIT_SSL_NO_VERIFY=yes git clone https://git.linux-mips.org/pub/scm/ralf/linux.git</code>.  Note that this disables the verification of the certificate thus leaving you vulnerable to the same attacks as plain http.
 +
** manually import the root certificate.  The preferable but also most complicated approach.
 +
* rsync The oldest git protocol, deprecated and supposed to eventually go away.  Suffers from a low probability race condition.  Its advantage is the lowest CPU usage on the server side.  Also some firewalls that don't allow git git protocol will allow rsync.  Not recommended for pulling or fetching.  Heck, it really should be considered the last alternative.  Rsync is not what in git parlance is called an intelligent transport which means that all new pack files on the server side will be transfered to the client even if only a single object from the pack would need to be transfered.  This however also means the bandwidth consumption can be excessive and is why rsync is no longer available as a git transport on linux-mips.org.
  
 
== Status of CVS to GIT conversion ==
 
== Status of CVS to GIT conversion ==
  
At this time linux.git and linux-malta.git trees are in active use. The MaltaRef_2_6 branch on linux-malta.git is a stable, tested kernel recommended for the [[Mips Malta]].  For the time being [[arcboot]] is still being maintained via CVS; all other archives are only historical.
+
All development has been moved to git.
  
 
== A note on branches in the kernel repository linux.git ==
 
== A note on branches in the kernel repository linux.git ==
Line 41: Line 64:
  
 
=== The queue ===
 
=== The queue ===
This repository at git://git.linux-mips.org/pub/scm/linux-queue.git contains all the patches which are queued up to be sent to Linus but currently are not being sent because we're currently in the stabilization phase for an upcoming release.  The repository is frequently being [[git:git-rebase|rebased]] to avoid piling up history and doing lots of merges which later on will just make the history unreadable.  Also patches may be updated, replaced or dropped.  It therefor is not recommendable to create your own branch based on this repository's queue branch&nbsp;- or at least you should not do unless you ''know'' what you're doing.  Of course as long as you're not updating your repository by pulling again from the queue branch you're also safe but if you want to fetch this queue branch anyway you want to use the "+" modifier before your refspec.  See the <tt>git-fetch</tt> or <tt>git-pull</tt> man pages for what this exactly means.
+
This repository at git://git.linux-mips.org/pub/scm/ralf/linux-queue.git contains all the patches which are queued up to be sent to Linus but currently are not being sent because we're currently in the stabilization phase for an upcoming release.  The repository is frequently being [[git:git-rebase|rebased]] to avoid piling up history and doing lots of merges which later on will just make the history unreadable.  Also patches may be updated, replaced or dropped.  It therefor is not recommendable to create your own branch based on this repository's queue branch&nbsp;- or at least you should not do unless you ''know'' what you're doing.  Of course as long as you're not updating your repository by pulling again from the queue branch you're also safe but if you want to fetch this queue branch anyway you want to use the "+" modifier before your refspec.  See the <tt>git-fetch</tt> or <tt>git-pull</tt> man pages for what this exactly means.
  
 
So why does the queue repository with all these restrictions exist then I hear you ask?  It's meant to allow people to peek into what patches will go to Linus soon and to create patches that will apply cleanly.  It's also a place where patches can ripen before they make it into the main repository.
 
So why does the queue repository with all these restrictions exist then I hear you ask?  It's meant to allow people to peek into what patches will go to Linus soon and to create patches that will apply cleanly.  It's also a place where patches can ripen before they make it into the main repository.
  
== Branches in the linux-malta.git repository ==
+
=== Linux/MIPS on kernel.org ===
The linux-malta.git repository is a superset of linux.git with the addition of two stable branches <tt>MaltaRef_2_6</tt> and <tt>MaltaRef_2_4</tt>. To use one of these stable branches after cloning the repository:
+
A copy of the Linux/MIPS kernel repository is also maintained on kernel.org. It is accessible at:
  
<pre>
+
* http://www.kernel.org/git/?p=linux/kernel/git/ralf/linux.git;a=summary (gitweb)
[ralf@dea linux-malta-git]$ git checkout MaltaRef_2_6
+
* git://git.kernel.org/pub/scm/linux/kernel/git/ralf/linux.git
</pre>
+
 
 +
Note that while kernel.org has a massive total of 2 gigabit network bandwidth it is currently very overloaded and due to the staged server system on kernel.org the copy of the git tree there may lag anywhere between 15min to over a day behind its linux-mips.org equivalent.
 +
 
 +
== The linux-malta.git repository ==
 +
The MaltaRef_2_6 branch on linux-malta.git used to be a stable, tested kernel recommended for the [[MIPS Malta]].  It contains a superset of linux.git with the addition of two stable branches <tt>MaltaRef_2_6</tt> and <tt>MaltaRef_2_4</tt>.  This repository has been obsoleted.  All changes now go straight into the main Linux repository and unless you're seeking a Malta hardened older kernel version you're probably better off if you use the <tt>linux.git</tt> repository.
  
 
== Checking out a tagged release with git ==
 
== Checking out a tagged release with git ==
Line 87: Line 114:
 
== Tags converted from the kernel CVS tree ==
 
== Tags converted from the kernel CVS tree ==
 
When converting the kernel CVS repository to git the tags were recreated by a script, not through conversion.  As such it is possible that the CVS tag and its equivalent git tag refer to slightly different versions; we haven't verified that.  Also, CVS didn't allow the "." character to be part of a tag name, so CVS tags used to look like <tt>linux_2_6_12</tt> - counterintuitive.  Git doesn't have such restrictions and so the tags name has become <tt>linux-2.6.12</tt>.
 
When converting the kernel CVS repository to git the tags were recreated by a script, not through conversion.  As such it is possible that the CVS tag and its equivalent git tag refer to slightly different versions; we haven't verified that.  Also, CVS didn't allow the "." character to be part of a tag name, so CVS tags used to look like <tt>linux_2_6_12</tt> - counterintuitive.  Git doesn't have such restrictions and so the tags name has become <tt>linux-2.6.12</tt>.
 
== Shallow kernel repository ==
 
The kernel repository is about 312MB.  This is the ''entire'' history of the Linux/MIPS project as far as still available dating back all the way to [[1994]], arguably quite a bit too much for most uses.  So there is now a second stripped down repository starting at 2.6.16.  It can be cloned with
 
 
<pre>
 
git clone git://ftp.linux-mips.org/pub/scm/linux-2.6.16.git linux.git
 
</pre>
 
  
 
== Disk space requirements for the kernel repository ==
 
== Disk space requirements for the kernel repository ==
As of this writing the size is approximately 63MB compared to 312MB for the full repository and the size saving of the shallow repository - obviously - comes at the price of the history before 2.6.15.    Other than that the tree is identical to the main git kernel repositoryAdditional 275MB are needed for checking out the repository plus another upto 85MB for building - even more for very large configurations, debugging etc.
+
As of <tt>linux-2.6.21</tt> the size is 308MB for the full repository.  Users of git&nbsp;1.5.0 and newer can use a git feature named ''shallow tree'' to prune the history. This can reduce the repository to as little as 112MBAn additional 297MB are needed for checking out the repository plus another upto 90MB for building - even more for very large configurations, debugging etc.
  
Using ''aggressive'' compression settings (see [[git:git-pack-objects|git-pack-objects]] manpage) on the order of 70MB can be saved for the full repository at the price of somewhat lower performance.  Linux-mips.org stopped doing so as it puts an inacceptable burden on the aging server machine.
+
Using ''aggressive'' compression settings (see [[git:git-pack-objects|git-pack-objects]] manpage) on the order of 70MB can be saved for the full repository at the price of somewhat lower performance.
  
 
== Gitweb ==
 
== Gitweb ==
Gitweb allows simple browsing of git repositories in a web browser.  Git web is available at http://www.linux-mips.org/git.
+
Gitweb allows simple browsing of git repositories in a web browser.  Git web is available at http://git.linux-mips.org.
  
 
== See also ==
 
== See also ==
 +
* http://www.linux-mips.org/git
 
* [http://kernel.org/pub/software/scm/git/docs/tutorial.html A short git tutorial] at [http://www.kernel.org kernel.org]
 
* [http://kernel.org/pub/software/scm/git/docs/tutorial.html A short git tutorial] at [http://www.kernel.org kernel.org]
 
* [http://git.or.cz The GIT homepage]
 
* [http://git.or.cz The GIT homepage]
 
* [[Mailing-patches]] How to email patches for submission into the kernel.
 
* [[Mailing-patches]] How to email patches for submission into the kernel.
 +
* http://www-128.ibm.com/developerworks/linux/library/l-vercon/index.html A nice article by IBM giving an overview what SCM is and several popular SCM systems.
 +
 +
 +
[[Category:git]]

Revision as of 18:14, 20 August 2012

::We will hereby start scouring the net for people who say git is hard to
understand and use, and just kill them. They clearly are just polluting
the gene pool.
Linus


All right, we're not quite that bad. Yet. These days the Linux world has largely switched to GIT as its SCM. Git is a fairly low-level thing, more the backing store of an SCM - or plumbing in Linus's words - than a full-blown SCM but it's growing up very quickly. Linux-mips.org has used CVS since 1997 and so naturally is a little more conservative in switching to a new tools as we don't want to drop all the history that's hidden in these trees.

GIT's rapid development has meant there is little high-level documentation, so here's a local page on WhatIsGit.

Accessing GIT repositories

The GIT repositories can be accessed by git://git.linux-mips.org/pub/scm. The http and rsync protocols are also supported but not recommended. Here are two example commands using the linux.git repository; you can substitute the name of another repository (see further below) for linux.git in the examples.

Cloning a repository

   git clone git://git.linux-mips.org/pub/scm/ralf/linux

This will create a new directory under the current working directory called linux. To use a different name specify it on the command line as an additional argument and it will become the target. Here is an alternate form of the command which creates a local directory called linux.git instead of linux.

   git clone git://git.linux-mips.org/pub/scm/ralf/linux linux.git

Updating a repository

When the repository was cloned the upstream URL was saved into the file .git/remotes/origin. This is the default location for git to pull updates. If you wish to modify the default URL you may edit that file directly.

From the top directory of your local repository (that is, the directory which contains the .git subdirectory) run the following command:

   git pull

This will pull updates from the repository and merge them into your local repository.

It is also possible to specify an alternate URL by specifying it on the command line.

   git pull git://git.linux-mips.org/pub/scm/ralf/linux

Typing in URLs can be tedious. You may create shortcuts to repeatedly used URLs by saving them in a remotes file. Alternate URLs may be saved in the .git/remotes/REMOTENAME file where the REMOTENAME string is the name that you wish to call that source location. Let's say that that you wanted to clone this long URL git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git but find that a lot to type repeatedly. If that is specified in a remotes file linus then the following command would pull from it. git pull linus

Which git protocol to use

Generally these days the git protocol (the git://) URLs) is the prefered protocol.

  • git Git's own protocol which tries to heavily optimize the amount of bandwidth used and thus is generally very efficient for updates. An issue with the git protocol is its use of TCP port 9418 which paranoid firewall admins may have blocked.
  • http Rather inefficient usage of bandwith and CPU but since http is generally enabled in firewalls it exists for those poor souls suffering from fascist firewall admins.
  • https More secure than http but otherwise pretty much the same thing. linux-mips.org uses SSL certificates from cacert.org and CACERT's root certificate is not included with most operating systems. A notable exception is Debian however. So if attempts to access a linux-mips.org repository via https fail with an error message about an invalid SSL certificate, you can:
    • use one of the other protocols listed in this section
    • disable the verification of the SSL certificate. On a UNIX or Linux type of system you do this by setting the GIT_SSL_NO_VERIFY variable, for example: GIT_SSL_NO_VERIFY=yes git clone https://git.linux-mips.org/pub/scm/ralf/linux.git. Note that this disables the verification of the certificate thus leaving you vulnerable to the same attacks as plain http.
    • manually import the root certificate. The preferable but also most complicated approach.
  • rsync The oldest git protocol, deprecated and supposed to eventually go away. Suffers from a low probability race condition. Its advantage is the lowest CPU usage on the server side. Also some firewalls that don't allow git git protocol will allow rsync. Not recommended for pulling or fetching. Heck, it really should be considered the last alternative. Rsync is not what in git parlance is called an intelligent transport which means that all new pack files on the server side will be transfered to the client even if only a single object from the pack would need to be transfered. This however also means the bandwidth consumption can be excessive and is why rsync is no longer available as a git transport on linux-mips.org.

Status of CVS to GIT conversion

All development has been moved to git.

A note on branches in the kernel repository linux.git

Following the recent changes in the Linux development model merges with Linus are now happening much more frequently and the difference between the git archives of Linus and linux-mips.org has become much smaller and is expected to shrink further. One side effect of this is that the heads of the branches will now pick up any kind of problem much faster than it used to be in the past. It therefore is suggested to people that don't want to fight bugs on this front to only use the tagged releases. This affects primarily the master branch on which the active 2.6 development is happening and to a lesser degree the linux-2.4 branch.

The queue

This repository at git://git.linux-mips.org/pub/scm/ralf/linux-queue.git contains all the patches which are queued up to be sent to Linus but currently are not being sent because we're currently in the stabilization phase for an upcoming release. The repository is frequently being rebased to avoid piling up history and doing lots of merges which later on will just make the history unreadable. Also patches may be updated, replaced or dropped. It therefor is not recommendable to create your own branch based on this repository's queue branch - or at least you should not do unless you know what you're doing. Of course as long as you're not updating your repository by pulling again from the queue branch you're also safe but if you want to fetch this queue branch anyway you want to use the "+" modifier before your refspec. See the git-fetch or git-pull man pages for what this exactly means.

So why does the queue repository with all these restrictions exist then I hear you ask? It's meant to allow people to peek into what patches will go to Linus soon and to create patches that will apply cleanly. It's also a place where patches can ripen before they make it into the main repository.

Linux/MIPS on kernel.org

A copy of the Linux/MIPS kernel repository is also maintained on kernel.org. It is accessible at:

Note that while kernel.org has a massive total of 2 gigabit network bandwidth it is currently very overloaded and due to the staged server system on kernel.org the copy of the git tree there may lag anywhere between 15min to over a day behind its linux-mips.org equivalent.

The linux-malta.git repository

The MaltaRef_2_6 branch on linux-malta.git used to be a stable, tested kernel recommended for the MIPS Malta. It contains a superset of linux.git with the addition of two stable branches MaltaRef_2_6 and MaltaRef_2_4. This repository has been obsoleted. All changes now go straight into the main Linux repository and unless you're seeking a Malta hardened older kernel version you're probably better off if you use the linux.git repository.

Checking out a tagged release with git

Bare git doesn't make this as easy as you'd like it to. So while git-checkout does support checking out the HEAD of a branch, tags are unfortunately not supported. But first, this is how to list the available tags:

[ralf@dea linux-git]$ ls .git/refs/tags
linux-1.1.68             linux-2.4.11       linux-2.5.53
[...]
linux-2.4.10             linux-2.6.14-rc1   linux-2.6.14
[ralf@dea linux-git]$

or

[ralf@dea linux-git]$ git tag -l
linux-1.1.68
linux-1.3.0
...

The actual list of tags is much longer. Okay, so let's assume we want to checkout the linux-2.6.14 release:

[ralf@dea linux-git]$ git checkout -b my-2.6.14 linux-2.6.14

What this exactly does is creating a new branch named my-2.6.14 at linux-2.6.14 and checking out it's HEAD.

Note that old versions of this page did recommend the use of git reset --hard. Don't. It will damage your repository by destructively removing anything newer than the version you're resetting to from your repository. See the git-reset man page for what git-reset really does.

Git pull and tags

Git considers tags something local. That means git clone will replicate all the tags but git pull will not pull new tags. Git 1.1 fixes this. Git-push however can be convinced to push tags by either explicitly listing them on the command line or the --all option.

Tags converted from the kernel CVS tree

When converting the kernel CVS repository to git the tags were recreated by a script, not through conversion. As such it is possible that the CVS tag and its equivalent git tag refer to slightly different versions; we haven't verified that. Also, CVS didn't allow the "." character to be part of a tag name, so CVS tags used to look like linux_2_6_12 - counterintuitive. Git doesn't have such restrictions and so the tags name has become linux-2.6.12.

Disk space requirements for the kernel repository

As of linux-2.6.21 the size is 308MB for the full repository. Users of git 1.5.0 and newer can use a git feature named shallow tree to prune the history. This can reduce the repository to as little as 112MB. An additional 297MB are needed for checking out the repository plus another upto 90MB for building - even more for very large configurations, debugging etc.

Using aggressive compression settings (see git-pack-objects manpage) on the order of 70MB can be saved for the full repository at the price of somewhat lower performance.

Gitweb

Gitweb allows simple browsing of git repositories in a web browser. Git web is available at http://git.linux-mips.org.

See also