Page size - Revision history

Ralf at 09:39, 12 October 2012

2012-10-12T09:39:33Z

Ralf at 14:54, 28 September 2012

2012-09-28T14:54:00Z

Ralf: /* Addressable virtual address space */

2012-09-27T21:48:20Z

‎Addressable virtual address space

Ralf: /* Addressable virtual address space */

2012-09-27T21:26:49Z

‎Addressable virtual address space

Ralf: /* Addressable virtual address space */

2012-09-27T21:21:44Z

‎Addressable virtual address space

Ralf: /* Accessible address space */

2012-09-27T19:27:55Z

‎Accessible address space

Ralf: /* Accessible address space */

2012-09-27T19:20:33Z

‎Accessible address space

Ralf: /* Accessible address space */

2012-09-27T18:55:42Z

‎Accessible address space

Ralf at 12:24, 27 September 2012

2012-09-27T12:24:16Z

Ralf: A few words on large page size and how the numbers work out.

2006-08-29T13:19:18Z

A few words on large page size and how the numbers work out.

New page

Since a while the Linux/MIPS kernel support page sizes other than 4kB.

* The [[R6000]] processor only supports 16kB page size.
* The [[R8000]] processor supports 8kB page size as it's smallest pagesize. It's also the only MIPS processor to support 8kB page size.
* Large compute workloads benefit sometimes dramatically from the increased ''TLB reach''. Downside: the memory overhead can be significant and at times prohibitive.
* Large pagesize eleminates cache aliases and the overhead and potencial for bugs associated with them.
* SPECbench numbers have been shown to be boosted significantly by 16kB pages and yet somewhat more by 64kB pages.

== Accessible address space ==
The size of the usable address space depends on the number of levels of the page tables, page size and the kernel model used. It always is 4GB for 32-bit kernels. Note that these 4GB are split into 2GB of userspace and 2GB of kernel addressing space. This split is hardwired in the MIPS architecture and (with the exception of the R6000) cannot be changed. The following table gives an overview over the situation on the 64-bit kernel.

{| {{PrettyTable}}
! page size !! levels !! virtual address space
|-
| 4kB || 1 || 21 bits / 4MB
|-
| 4kB || 2 || 30 bits / 1GB
|-
| 4kB || 3 || 39 bits / 256GB
|-
| 4kB || 4 || 48 bits / 256TB
|-
| 8kB || 1 || 23 bits / 8MB
|-
| 8kB || 2 || 33 bits / 8GB
|-
| 8kB || 3 || 43 bits / 8TB
|-
| 8kB || 4 || 53 bits / 8PB
|-
| 16kB || 1 || 25 bits / 32MB
|-
| 16kB || 2 || 36 bits / 64GB
|-
| 16kB || 3 || 47 bits / 128TB
|-
| 16kB || 4 || 58 bits / 256PB
|-
| 64kB || 1 || 29 bits / 512MB
|-
| 64kB || 2 || 42 bits / 4TB
|-
| 64kB || 3 || 55 bits / 32PB
|-
| 64kB || 4 || 68 bits / 256 EB
|}

This table is mostly meant to show how the numbers work out. In reality there are a few constraints, so it should be taken with a grain of salt:
* The largest virtual address space supported by any MIPS processor is 48bits on the old and rather odd R8000 processor. Typical 64-bit MIPS processors support 36-bit, 40-bit or 44-bit address spaces but other sizes are permitted by the [[MIPS64]] architecture.
* Address spaces of less than 2GB are not likely to be very useful either.
* Only 3-level tables are supported by 64-bit MIPS kernels as of Linux 2.6.18.
* As of 2.6.18 16kB pagesize is supported for 64-bit kernels but experimental for the 32-bit kernel. 64kB is experimental.

@@ Line 49: / Line 49: @@
 * The largest virtual address space supported by any MIPS processor is 48bits on the old and rather odd R8000 processor.  Typical 64-bit MIPS processors support 36-bit, 40-bit or 44-bit address spaces but other sizes are permitted by the [[MIPS64]] architecture.
 * Address spaces of less than 2GB are not likely to be very useful either.
-* Only 3-level tables are supported by 64-bit MIPS kernels.  The exception are 64kB pages where the resulting 55&nbsp;bits&nbsp/ 32PB address space would be so large that cropping it to 42&nbsp;bits&nbsp;/ 4TB appeared more practical for the time being.
+* Only 3-level tables are supported by 64-bit MIPS kernels.  The exception are 64kB pages where the resulting 55&nbsp;bits&nbsp; 32PB address space would be so large that cropping it to 42&nbsp;bits&nbsp;/ 4TB appeared more practical for the time being.
 * 32-bit kernels only support 4kB and 16kB page size

@@ Line 31: / Line 31: @@
 * Address spaces of less than 2GB are not likely to be very useful either.
 * Only 3-level tables are supported by 64-bit MIPS kernels.  The exception are 64kB pages where the resulting 55&nbsp;bits&nbsp/ 32PB address space would be so large that cropping it to 42&nbsp;bits&nbsp;/ 4TB appeared more practical for the time being.
-* As of 2.6.18 16kB pagesize is supported for 64-bit kernels but experimental for the 32-bit kernel.  64kB is experimental.
+* 32-bit kernels only support 4kB and 16kB page size
 == Software support ==

@@ Line 19: / Line 19: @@
 | 16kB      || 25 bits / 32MB  || 36 bits / 64GB  || style="background-color: #ffe387;" | 47 bits / 128TB || 58 bits / 256PB
 |-
-| 32kB      || 27 bits / 128MB || 39 bits / 512GB || style="background-color: #ffe387;" | 51 bits / 2PB   || 64 bits / 4EB
+| 32kB      || 27 bits / 128MB || 39 bits / 512GB || style="background-color: #ffe387;" | 51 bits / 2PB   || style="background-color: #ffbbbb;" | 63 bits / 8EB
 |-
-| 64kB      || 29 bits / 512MB || 42 bits / 4TB   || style="background-color: #ffe387;" | 55 bits / 32PB  || 68 bits / 256EB
+| 64kB      || 29 bits / 512MB || style="background-color: #ffe387;" | 42 bits / 4TB   || 55 bits / 32PB  || style="background-color: #ffbbbb;" | 68 bits / 256EB
 |}
-This table is mostly meant to show how the numbers work out.  In reality there are a few constraints, so it should be taken with a grain of salt:
+The combinations marked red are nonsenical because <tt>XUSEG</tt> limits the user address space to 62&nbsp;bits&nbsp;/ 4EB.
 * For 3&nbsp;level page tableswith 4kB pages Linux uses two pages for the page directory so the actual virtual address space capacity is 40&nbsp;bits&nbsp;/ 1TB.
 * The largest virtual address space supported by any MIPS processor is 48bits on the old and rather odd R8000 processor.  Typical 64-bit MIPS processors support 36-bit, 40-bit or 44-bit address spaces but other sizes are permitted by the [[MIPS64]] architecture.
 * Address spaces of less than 2GB are not likely to be very useful either.
-* Only 3-level tables are supported by 64-bit MIPS kernels.
+* Only 3-level tables are supported by 64-bit MIPS kernels.  The exception are 64kB pages where the resulting 55&nbsp;bits&nbsp/ 32PB address space would be so large that cropping it to 42&nbsp;bits&nbsp;/ 4TB appeared more practical for the time being.
 * As of 2.6.18 16kB pagesize is supported for 64-bit kernels but experimental for the 32-bit kernel.  64kB is experimental.

@@ Line 11: / Line 11: @@
 {| {{PrettyTable}}
-! Page Size !! 1 Level         !! 2 Level         !! 3 Levels        !! 4 Levels
+! Page Size !! 1 Level         !! 2 Level         !! style="background-color: #ffe387;" | 3 Levels        !! 4 Levels
 |-
-| 4kB       || 21 bits / 2MB   || 30 bits / 1GB   || 39 bits / 512GB || 48 bits / 256TB
+| 4kB       || 21 bits / 2MB   || 30 bits / 1GB   || style="background-color: #ffe387;" | 39 bits / 512GB || 48 bits / 256TB
 |-
-| 8kB       || 23 bits / 8MB   || 33 bits / 8GB   || 43 bits / 8TB   || 53 bits / 8PB
+| 8kB       || 23 bits / 8MB   || 33 bits / 8GB   || style="background-color: #ffe387;" | 43 bits / 8TB   || 53 bits / 8PB
 |-
-| 16kB      || 25 bits / 32MB  || 36 bits / 64GB  || 47 bits / 128TB || 58 bits / 256PB
+| 16kB      || 25 bits / 32MB  || 36 bits / 64GB  || style="background-color: #ffe387;" | 47 bits / 128TB || 58 bits / 256PB
 |-
-| 32kB      || 27 bits / 128MB || 39 bits / 512GB || 51 bits / 2PB   || 64 bits / 4EB
+| 32kB      || 27 bits / 128MB || 39 bits / 512GB || style="background-color: #ffe387;" | 51 bits / 2PB   || 64 bits / 4EB
 |-
-| 64kB      || 29 bits / 512MB || 42 bits / 4TB   || 55 bits / 32PB  || 68 bits / 256EB
+| 64kB      || 29 bits / 512MB || 42 bits / 4TB   || style="background-color: #ffe387;" | 55 bits / 32PB  || 68 bits / 256EB
 |}

← Older revision		Revision as of 12:24, 27 September 2012
Line 51:		Line 51:
	* Only 3-level tables are supported by 64-bit MIPS kernels as of Linux 2.6.18.		* Only 3-level tables are supported by 64-bit MIPS kernels as of Linux 2.6.18.
	* As of 2.6.18 16kB pagesize is supported for 64-bit kernels but experimental for the 32-bit kernel. 64kB is experimental.		* As of 2.6.18 16kB pagesize is supported for 64-bit kernels but experimental for the 32-bit kernel. 64kB is experimental.
		+
		+	== Software support ==
		+	=== Hugetlbfs ===
		+	Hugetlbfs is a special magic filesystem. Any file mmapped from hugetlbfs will use huge pages. The problem with hugetlbfs is that it software will need to be modified to use it and memory to be used for huge pages with hugetlbfs needs to be put aside manually.
		+
		+	Cavium's SDK is able to put away with the need to modify software by using an special preload library that wraps <tt>malloc</tt>. This method works for most but not all applications.
		+
		+	=== Transparent Huge Pages ===
		+	[[Transparent Huge Pages]] allows normal unmodified software to use huge pages. Other than possibly having to enable THP support there is no administrative activity required. The price for this is a slight overhead.

@@ Line 1: / Line 1: @@
 Since a while the Linux/MIPS kernel support page sizes other than 4kB.
-* The [[R6000]] processor only supports 16kB page size.
+* Large compute workloads benefit sometimes dramatically from the increased ''TLB reach''.  Downside: the memory overhead can be significant and at times prohibitive for small systems or workloads that use a large number of small memory mappings.
-* The [[R8000]] processor supports 8kB page size as it's smallest pagesize.  It's also the only MIPS processor to support 8kB page size.
+* Large page size eleminates cache aliases and the overhead and potential for bugs associated with them.
 * Large compute workloads benefit sometimes dramatically from the increased ''TLB reach''.  Downside: the memory overhead can be significant and at times prohibitive.
-* Large pagesize eleminates cache aliases and the overhead and potencial for bugs associated with them.
 * SPECbench numbers have been shown to be boosted significantly by 16kB pages and yet somewhat more by 64kB pages.
 == Addressable virtual address space ==

@@ Line 7: / Line 7: @@
 * SPECbench numbers have been shown to be boosted significantly by 16kB pages and yet somewhat more by 64kB pages.
-== Accessible address space ==
+== Addressable virtual address space ==
 The size of the usable address space depends on the number of levels of the page tables, page size and the kernel model used.  It always is 4GB for 32-bit kernels.  Note that these 4GB are split into 2GB of userspace and 2GB of kernel addressing space.  This split is hardwired in the MIPS architecture and (with the exception of the R6000) cannot be changed.  The following table gives an overview over the situation on the 64-bit kernel.

@@ Line 11: / Line 11: @@
 {| {{PrettyTable}}
-! page size !! levels !! virtual address space
+! Page Size !! 1 Level         !! 2 Level         !! 3 Levels        !! 4 Levels
 |-
-| 4kB || 1 || 21 bits / 4MB
+| 4kB       || 21 bits / 2MB   || 30 bits / 1GB   || 39 bits / 512GB || 48 bits / 256TB
 |-
-| 4kB || 2 || 30 bits / 1GB
+| 8kB       || 23 bits / 8MB   || 33 bits / 8GB   || 43 bits / 8TB   || 53 bits / 8PB
 |-
-| 4kB || 3 || 39 bits / 256GB
+| 16kB      || 25 bits / 32MB  || 36 bits / 64GB  || 47 bits / 128TB || 58 bits / 256PB
 |-
-| 4kB || 4 || 48 bits / 256TB
+| 32kB      || 27 bits / 128MB || 39 bits / 512GB || 51 bits / 2PB   || 64 bits / 4EB
 |-
-| 8kB || 1 || 23 bits / 8MB
+| 64kB      || 29 bits / 512MB || 42 bits / 4TB   || 55 bits / 32PB  || 68 bits / 256EB
-|-
-| 8kB || 2 || 33 bits / 8GB
-|-
-| 8kB || 3 || 43 bits / 8TB
-|-
-| 8kB || 4 || 53 bits / 8PB
-|-
-| 16kB || 1 || 25 bits / 32MB
-|-
-| 16kB || 2 || 36 bits / 64GB
-|-
-| 16kB || 3 || 47 bits / 128TB
-|-
-| 16kB || 4 || 58 bits / 256PB
-|-
-| 64kB || 1 || 29 bits / 512MB
-|-
-| 64kB || 2 || 42 bits / 4TB
-|-
-| 64kB || 3 || 55 bits / 32PB
-|-
-| 64kB || 4 || 68 bits / 256 EB
 |}
 This table is mostly meant to show how the numbers work out.  In reality there are a few constraints, so it should be taken with a grain of salt:
 * The largest virtual address space supported by any MIPS processor is 48bits on the old and rather odd R8000 processor.  Typical 64-bit MIPS processors support 36-bit, 40-bit or 44-bit address spaces but other sizes are permitted by the [[MIPS64]] architecture.
 * Address spaces of less than 2GB are not likely to be very useful either.
-* Only 3-level tables are supported by 64-bit MIPS kernels as of Linux&nbsp;2.6.18.
+* Only 3-level tables are supported by 64-bit MIPS kernels.
 * As of 2.6.18 16kB pagesize is supported for 64-bit kernels but experimental for the 32-bit kernel.  64kB is experimental.