Skip to content
Snippets Groups Projects
  • Andy Pan's avatar
    970b1c04
    os: increase the amount of data transfer for sendfile(2) to reduce syscalls · 970b1c04
    Andy Pan authored
    For the moment, Go calls sendfile(2) to transfer at most 4MB at a time
    while sendfile(2) actually allows a larger amount of data on one call.
    To reduce system calls of sendfile(2) during data copying, we should
    specify the number of bytes to copy as large as possible.
    
    This optimization is especially advantageous for bulky file-to-file copies,
    it would lead to a performance boost, the magnitude of this performance
    increase may not be very exciting, but it can also cut down the CPU overhead
    by decreasing the number of system calls.
    
    This is also how we've done in sendfile_windows.go with TransmitFile.
    
    goos: linux
    goarch: amd64
    pkg: os
    cpu: DO-Premium-AMD
               │    old     │                new                │
               │   sec/op   │   sec/op    vs base               │
    SendFile-8   1.135 ± 4%   1.052 ± 3%  -7.24% (p=0.000 n=10)
    
               │     old      │                 new                 │
               │     B/s      │     B/s       vs base               │
    SendFile-8   902.5Mi ± 4%   973.0Mi ± 3%  +7.81% (p=0.000 n=10)
    
               │    old     │              new               │
               │    B/op    │    B/op     vs base            │
    SendFile-8   272.0 ± 0%   272.0 ± 0%  ~ (p=1.000 n=10) ¹
    ¹ all samples are equal
    
               │    old     │              new               │
               │ allocs/op  │ allocs/op   vs base            │
    SendFile-8   20.00 ± 0%   20.00 ± 0%  ~ (p=1.000 n=10) ¹
    ¹ all samples are equal
    
    Change-Id: Ib4d4c6bc693e23db24697363b29226f0c9776bb0
    Reviewed-on: https://go-review.googlesource.com/c/go/+/605235
    
    
    LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
    Auto-Submit: Ian Lance Taylor <iant@golang.org>
    TryBot-Result: Gopher Robot <gobot@golang.org>
    Reviewed-by: default avatarJorropo <jorropo.pgm@gmail.com>
    Run-TryBot: Andy Pan <panjf2000@gmail.com>
    Reviewed-by: default avatarIan Lance Taylor <iant@google.com>
    Reviewed-by: default avatarCarlos Amedee <carlos@golang.org>
    970b1c04
    History
    os: increase the amount of data transfer for sendfile(2) to reduce syscalls
    Andy Pan authored
    For the moment, Go calls sendfile(2) to transfer at most 4MB at a time
    while sendfile(2) actually allows a larger amount of data on one call.
    To reduce system calls of sendfile(2) during data copying, we should
    specify the number of bytes to copy as large as possible.
    
    This optimization is especially advantageous for bulky file-to-file copies,
    it would lead to a performance boost, the magnitude of this performance
    increase may not be very exciting, but it can also cut down the CPU overhead
    by decreasing the number of system calls.
    
    This is also how we've done in sendfile_windows.go with TransmitFile.
    
    goos: linux
    goarch: amd64
    pkg: os
    cpu: DO-Premium-AMD
               │    old     │                new                │
               │   sec/op   │   sec/op    vs base               │
    SendFile-8   1.135 ± 4%   1.052 ± 3%  -7.24% (p=0.000 n=10)
    
               │     old      │                 new                 │
               │     B/s      │     B/s       vs base               │
    SendFile-8   902.5Mi ± 4%   973.0Mi ± 3%  +7.81% (p=0.000 n=10)
    
               │    old     │              new               │
               │    B/op    │    B/op     vs base            │
    SendFile-8   272.0 ± 0%   272.0 ± 0%  ~ (p=1.000 n=10) ¹
    ¹ all samples are equal
    
               │    old     │              new               │
               │ allocs/op  │ allocs/op   vs base            │
    SendFile-8   20.00 ± 0%   20.00 ± 0%  ~ (p=1.000 n=10) ¹
    ¹ all samples are equal
    
    Change-Id: Ib4d4c6bc693e23db24697363b29226f0c9776bb0
    Reviewed-on: https://go-review.googlesource.com/c/go/+/605235
    
    
    LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
    Auto-Submit: Ian Lance Taylor <iant@golang.org>
    TryBot-Result: Gopher Robot <gobot@golang.org>
    Reviewed-by: default avatarJorropo <jorropo.pgm@gmail.com>
    Run-TryBot: Andy Pan <panjf2000@gmail.com>
    Reviewed-by: default avatarIan Lance Taylor <iant@google.com>
    Reviewed-by: default avatarCarlos Amedee <carlos@golang.org>
Code owners
Assign users and groups as approvers for specific file changes. Learn more.