1 // Copyright 2009 The Go Authors. All rights reserved.
2 // Use of this source code is governed by a BSD-style
3 // license that can be found in the LICENSE file.
9 "runtime/internal/atomic"
17 // Beyond indicating the general state of a G, the G status
18 // acts like a lock on the goroutine's stack (and hence its
19 // ability to execute user code).
21 // If you add to this list, add to the list
22 // of "okay during garbage collection" status
25 // TODO(austin): The _Gscan bit could be much lighter-weight.
26 // For example, we could choose not to run _Gscanrunnable
27 // goroutines found in the run queue, rather than CAS-looping
28 // until they become _Grunnable. And transitions like
29 // _Gscanwaiting -> _Gscanrunnable are actually okay because
30 // they don't affect stack ownership.
32 // _Gidle means this goroutine was just allocated and has not
33 // yet been initialized.
36 // _Grunnable means this goroutine is on a run queue. It is
37 // not currently executing user code. The stack is not owned.
40 // _Grunning means this goroutine may execute user code. The
41 // stack is owned by this goroutine. It is not on a run queue.
42 // It is assigned an M and a P (g.m and g.m.p are valid).
45 // _Gsyscall means this goroutine is executing a system call.
46 // It is not executing user code. The stack is owned by this
47 // goroutine. It is not on a run queue. It is assigned an M.
50 // _Gwaiting means this goroutine is blocked in the runtime.
51 // It is not executing user code. It is not on a run queue,
52 // but should be recorded somewhere (e.g., a channel wait
53 // queue) so it can be ready()d when necessary. The stack is
54 // not owned *except* that a channel operation may read or
55 // write parts of the stack under the appropriate channel
56 // lock. Otherwise, it is not safe to access the stack after a
57 // goroutine enters _Gwaiting (e.g., it may get moved).
60 // _Gmoribund_unused is currently unused, but hardcoded in gdb
62 _Gmoribund_unused // 5
64 // _Gdead means this goroutine is currently unused. It may be
65 // just exited, on a free list, or just being initialized. It
66 // is not executing user code. It may or may not have a stack
67 // allocated. The G and its stack (if any) are owned by the M
68 // that is exiting the G or that obtained the G from the free
72 // _Genqueue_unused is currently unused.
75 // _Gcopystack means this goroutine's stack is being moved. It
76 // is not executing user code and is not on a run queue. The
77 // stack is owned by the goroutine that put it in _Gcopystack.
80 // _Gpreempted means this goroutine stopped itself for a
81 // suspendG preemption. It is like _Gwaiting, but nothing is
82 // yet responsible for ready()ing it. Some suspendG must CAS
83 // the status to _Gwaiting to take responsibility for
87 // _Gscan combined with one of the above states other than
88 // _Grunning indicates that GC is scanning the stack. The
89 // goroutine is not executing user code and the stack is owned
90 // by the goroutine that set the _Gscan bit.
92 // _Gscanrunning is different: it is used to briefly block
93 // state transitions while GC signals the G to scan its own
94 // stack. This is otherwise like _Grunning.
96 // atomicstatus&~Gscan gives the state the goroutine will
97 // return to when the scan completes.
99 _Gscanrunnable = _Gscan + _Grunnable // 0x1001
100 _Gscanrunning = _Gscan + _Grunning // 0x1002
101 _Gscansyscall = _Gscan + _Gsyscall // 0x1003
102 _Gscanwaiting = _Gscan + _Gwaiting // 0x1004
103 _Gscanpreempted = _Gscan + _Gpreempted // 0x1009
109 // _Pidle means a P is not being used to run user code or the
110 // scheduler. Typically, it's on the idle P list and available
111 // to the scheduler, but it may just be transitioning between
114 // The P is owned by the idle list or by whatever is
115 // transitioning its state. Its run queue is empty.
118 // _Prunning means a P is owned by an M and is being used to
119 // run user code or the scheduler. Only the M that owns this P
120 // is allowed to change the P's status from _Prunning. The M
121 // may transition the P to _Pidle (if it has no more work to
122 // do), _Psyscall (when entering a syscall), or _Pgcstop (to
123 // halt for the GC). The M may also hand ownership of the P
124 // off directly to another M (e.g., to schedule a locked G).
127 // _Psyscall means a P is not running user code. It has
128 // affinity to an M in a syscall but is not owned by it and
129 // may be stolen by another M. This is similar to _Pidle but
130 // uses lightweight transitions and maintains M affinity.
132 // Leaving _Psyscall must be done with a CAS, either to steal
133 // or retake the P. Note that there's an ABA hazard: even if
134 // an M successfully CASes its original P back to _Prunning
135 // after a syscall, it must understand the P may have been
136 // used by another M in the interim.
139 // _Pgcstop means a P is halted for STW and owned by the M
140 // that stopped the world. The M that stopped the world
141 // continues to use its P, even in _Pgcstop. Transitioning
142 // from _Prunning to _Pgcstop causes an M to release its P and
145 // The P retains its run queue and startTheWorld will restart
146 // the scheduler on Ps with non-empty run queues.
149 // _Pdead means a P is no longer used (GOMAXPROCS shrank). We
150 // reuse Ps if GOMAXPROCS increases. A dead P is mostly
151 // stripped of its resources, though a few things remain
152 // (e.g., trace buffers).
156 // Mutual exclusion locks. In the uncontended case,
157 // as fast as spin locks (just a few user-level instructions),
158 // but on the contention path they sleep in the kernel.
159 // A zeroed Mutex is unlocked (no need to initialize each lock).
160 // Initialization is helpful for static lock ranking, but not required.
162 // Empty struct if lock ranking is disabled, otherwise includes the lock rank
164 // Futex-based impl treats it as uint32 key,
165 // while sema-based impl as M* waitm.
166 // Used to be a union, but unions break precise GC.
170 // sleep and wakeup on one-time events.
171 // before any calls to notesleep or notewakeup,
172 // must call noteclear to initialize the Note.
173 // then, exactly one thread can call notesleep
174 // and exactly one thread can call notewakeup (once).
175 // once notewakeup has been called, the notesleep
176 // will return. future notesleep will return immediately.
177 // subsequent noteclear must be called only after
178 // previous notesleep has returned, e.g. it's disallowed
179 // to call noteclear straight after notewakeup.
181 // notetsleep is like notesleep but wakes up after
182 // a given number of nanoseconds even if the event
183 // has not yet happened. if a goroutine uses notetsleep to
184 // wake up early, it must wait to call noteclear until it
185 // can be sure that no other goroutine is calling
188 // notesleep/notetsleep are generally called on g0,
189 // notetsleepg is similar to notetsleep but is called on user g.
191 // Futex-based impl treats it as uint32 key,
192 // while sema-based impl as M* waitm.
193 // Used to be a union, but unions break precise GC.
197 type funcval struct {
199 // variable-size, fn-specific data here
212 func efaceOf(ep *interface{}) *eface {
213 return (*eface)(unsafe.Pointer(ep))
216 // The guintptr, muintptr, and puintptr are all used to bypass write barriers.
217 // It is particularly important to avoid write barriers when the current P has
218 // been released, because the GC thinks the world is stopped, and an
219 // unexpected write barrier would not be synchronized with the GC,
220 // which can lead to a half-executed write barrier that has marked the object
221 // but not queued it. If the GC skips the object and completes before the
222 // queuing can occur, it will incorrectly free the object.
224 // We tried using special assignment functions invoked only when not
225 // holding a running P, but then some updates to a particular memory
226 // word went through write barriers and some did not. This breaks the
227 // write barrier shadow checking mode, and it is also scary: better to have
228 // a word that is completely ignored by the GC than to have one for which
229 // only a few updates are ignored.
231 // Gs and Ps are always reachable via true pointers in the
232 // allgs and allp lists or (during allocation before they reach those lists)
233 // from stack variables.
235 // Ms are always reachable via true pointers either from allm or
236 // freem. Unlike Gs and Ps we do free Ms, so it's important that
237 // nothing ever hold an muintptr across a safe point.
239 // A guintptr holds a goroutine pointer, but typed as a uintptr
240 // to bypass write barriers. It is used in the Gobuf goroutine state
241 // and in scheduling lists that are manipulated without a P.
243 // The Gobuf.g goroutine pointer is almost always updated by assembly code.
244 // In one of the few places it is updated by Go code - func save - it must be
245 // treated as a uintptr to avoid a write barrier being emitted at a bad time.
246 // Instead of figuring out how to emit the write barriers missing in the
247 // assembly manipulation, we change the type of the field to uintptr,
248 // so that it does not require write barriers at all.
250 // Goroutine structs are published in the allg list and never freed.
251 // That will keep the goroutine structs from being collected.
252 // There is never a time that Gobuf.g's contain the only references
253 // to a goroutine: the publishing of the goroutine in allg comes first.
254 // Goroutine pointers are also kept in non-GC-visible places like TLS,
255 // so I can't see them ever moving. If we did want to start moving data
256 // in the GC, we'd need to allocate the goroutine structs from an
257 // alternate arena. Using guintptr doesn't make that problem any worse.
258 type guintptr uintptr
261 func (gp guintptr) ptr() *g { return (*g)(unsafe.Pointer(gp)) }
264 func (gp *guintptr) set(g *g) { *gp = guintptr(unsafe.Pointer(g)) }
267 func (gp *guintptr) cas(old, new guintptr) bool {
268 return atomic.Casuintptr((*uintptr)(unsafe.Pointer(gp)), uintptr(old), uintptr(new))
271 // setGNoWB performs *gp = new without a write barrier.
272 // For times when it's impractical to use a guintptr.
275 func setGNoWB(gp **g, new *g) {
276 (*guintptr)(unsafe.Pointer(gp)).set(new)
279 type puintptr uintptr
282 func (pp puintptr) ptr() *p { return (*p)(unsafe.Pointer(pp)) }
285 func (pp *puintptr) set(p *p) { *pp = puintptr(unsafe.Pointer(p)) }
287 // muintptr is a *m that is not tracked by the garbage collector.
289 // Because we do free Ms, there are some additional constrains on
292 // 1. Never hold an muintptr locally across a safe point.
294 // 2. Any muintptr in the heap must be owned by the M itself so it can
295 // ensure it is not in use when the last true *m is released.
296 type muintptr uintptr
299 func (mp muintptr) ptr() *m { return (*m)(unsafe.Pointer(mp)) }
302 func (mp *muintptr) set(m *m) { *mp = muintptr(unsafe.Pointer(m)) }
304 // setMNoWB performs *mp = new without a write barrier.
305 // For times when it's impractical to use an muintptr.
308 func setMNoWB(mp **m, new *m) {
309 (*muintptr)(unsafe.Pointer(mp)).set(new)
313 // The offsets of sp, pc, and g are known to (hard-coded in) libmach.
315 // ctxt is unusual with respect to GC: it may be a
316 // heap-allocated funcval, so GC needs to track it, but it
317 // needs to be set and cleared from assembly, where it's
318 // difficult to have write barriers. However, ctxt is really a
319 // saved, live register, and we only ever exchange it between
320 // the real register and the gobuf. Hence, we treat it as a
321 // root during stack scanning, which means assembly that saves
322 // and restores it doesn't need write barriers. It's still
323 // typed as a pointer so that any other writes from Go get
331 bp uintptr // for framepointer-enabled architectures
334 // sudog represents a g in a wait list, such as for sending/receiving
337 // sudog is necessary because the g ↔ synchronization object relation
338 // is many-to-many. A g can be on many wait lists, so there may be
339 // many sudogs for one g; and many gs may be waiting on the same
340 // synchronization object, so there may be many sudogs for one object.
342 // sudogs are allocated from a special pool. Use acquireSudog and
343 // releaseSudog to allocate and free them.
345 // The following fields are protected by the hchan.lock of the
346 // channel this sudog is blocking on. shrinkstack depends on
347 // this for sudogs involved in channel ops.
353 elem unsafe.Pointer // data element (may point to stack)
355 // The following fields are never accessed concurrently.
356 // For channels, waitlink is only accessed by g.
357 // For semaphores, all fields (including the ones above)
358 // are only accessed when holding a semaRoot lock.
364 // isSelect indicates g is participating in a select, so
365 // g.selectDone must be CAS'd to win the wake-up race.
368 // success indicates whether communication over channel c
369 // succeeded. It is true if the goroutine was awoken because a
370 // value was delivered over channel c, and false if awoken
371 // because c was closed.
374 parent *sudog // semaRoot binary tree
375 waitlink *sudog // g.waiting list or semaRoot
376 waittail *sudog // semaRoot
380 type libcall struct {
382 n uintptr // number of parameters
383 args uintptr // parameters
384 r1 uintptr // return values
386 err uintptr // error number
389 // Stack describes a Go execution stack.
390 // The bounds of the stack are exactly [lo, hi),
391 // with no implicit data structures on either side.
397 // heldLockInfo gives info on a held lock and the rank of that lock
398 type heldLockInfo struct {
405 // stack describes the actual stack memory: [stack.lo, stack.hi).
406 // stackguard0 is the stack pointer compared in the Go stack growth prologue.
407 // It is stack.lo+StackGuard normally, but can be StackPreempt to trigger a preemption.
408 // stackguard1 is the stack pointer compared in the C stack growth prologue.
409 // It is stack.lo+StackGuard on g0 and gsignal stacks.
410 // It is ~0 on other goroutine stacks, to trigger a call to morestackc (and crash).
411 stack stack // offset known to runtime/cgo
412 stackguard0 uintptr // offset known to liblink
413 stackguard1 uintptr // offset known to liblink
415 _panic *_panic // innermost panic - offset known to liblink
416 _defer *_defer // innermost defer
417 m *m // current m; offset known to arm liblink
419 syscallsp uintptr // if status==Gsyscall, syscallsp = sched.sp to use during gc
420 syscallpc uintptr // if status==Gsyscall, syscallpc = sched.pc to use during gc
421 stktopsp uintptr // expected sp at top of stack, to check in traceback
422 // param is a generic pointer parameter field used to pass
423 // values in particular contexts where other storage for the
424 // parameter would be difficult to find. It is currently used
426 // 1. When a channel operation wakes up a blocked goroutine, it sets param to
427 // point to the sudog of the completed blocking operation.
428 // 2. By gcAssistAlloc1 to signal back to its caller that the goroutine completed
429 // the GC cycle. It is unsafe to do so in any other way, because the goroutine's
430 // stack may have moved in the meantime.
431 // 3. By debugCallWrap to pass parameters to a new goroutine because allocating a
432 // closure in the runtime is forbidden.
435 stackLock uint32 // sigprof/scang lock; TODO: fold in to atomicstatus
438 waitsince int64 // approx time when the g become blocked
439 waitreason waitReason // if status==Gwaiting
441 preempt bool // preemption signal, duplicates stackguard0 = stackpreempt
442 preemptStop bool // transition to _Gpreempted on preemption; otherwise, just deschedule
443 preemptShrink bool // shrink stack at synchronous safe point
445 // asyncSafePoint is set if g is stopped at an asynchronous
446 // safe point. This means there are frames on the stack
447 // without precise pointer information.
450 paniconfault bool // panic (instead of crash) on unexpected fault address
451 gcscandone bool // g has scanned stack; protected by _Gscan bit in status
452 throwsplit bool // must not split stack
453 // activeStackChans indicates that there are unlocked channels
454 // pointing into this goroutine's stack. If true, stack
455 // copying needs to acquire channel locks to protect these
456 // areas of the stack.
457 activeStackChans bool
458 // parkingOnChan indicates that the goroutine is about to
459 // park on a chansend or chanrecv. Used to signal an unsafe point
460 // for stack shrinking. It's a boolean value, but is updated atomically.
463 raceignore int8 // ignore race detection events
464 sysblocktraced bool // StartTrace has emitted EvGoInSyscall about this goroutine
465 tracking bool // whether we're tracking this G for sched latency statistics
466 trackingSeq uint8 // used to decide whether to track this G
467 runnableStamp int64 // timestamp of when the G last became runnable, only used when tracking
468 runnableTime int64 // the amount of time spent runnable, cleared when running, only used when tracking
469 sysexitticks int64 // cputicks when syscall has returned (for tracing)
470 traceseq uint64 // trace event sequencer
471 tracelastp puintptr // last P emitted an event for this goroutine
478 gopc uintptr // pc of go statement that created this goroutine
479 ancestors *[]ancestorInfo // ancestor information goroutine(s) that created this goroutine (only used if debug.tracebackancestors)
480 startpc uintptr // pc of goroutine function
482 waiting *sudog // sudog structures this g is waiting on (that have a valid elem ptr); in lock order
483 cgoCtxt []uintptr // cgo traceback context
484 labels unsafe.Pointer // profiler labels
485 timer *timer // cached timer for time.Sleep
486 selectDone uint32 // are we participating in a select and did someone win the race?
490 // gcAssistBytes is this G's GC assist credit in terms of
491 // bytes allocated. If this is positive, then the G has credit
492 // to allocate gcAssistBytes bytes without assisting. If this
493 // is negative, then the G must correct this by performing
494 // scan work. We track this in bytes to make it fast to update
495 // and check for debt in the malloc hot path. The assist ratio
496 // determines how this corresponds to scan work debt.
500 // gTrackingPeriod is the number of transitions out of _Grunning between
501 // latency tracking runs.
502 const gTrackingPeriod = 8
505 // tlsSlots is the number of pointer-sized slots reserved for TLS on some platforms,
508 tlsSize = tlsSlots * goarch.PtrSize
512 g0 *g // goroutine with scheduling stack
513 morebuf gobuf // gobuf arg to morestack
514 divmod uint32 // div/mod denominator for arm - known to liblink
516 // Fields not known to debuggers.
517 procid uint64 // for debuggers, but offset not hard-coded
518 gsignal *g // signal-handling g
519 goSigStack gsignalStack // Go-allocated signal handling stack
520 sigmask sigset // storage for saved signal mask
521 tls [tlsSlots]uintptr // thread-local storage (for x86 extern register)
523 curg *g // current running goroutine
524 caughtsig guintptr // goroutine running during fatal signal
525 p puintptr // attached p for executing go code (nil if not executing go code)
527 oldp puintptr // the p that was attached before executing a syscall
531 preemptoff string // if != "", keep curg running on this m
535 spinning bool // m is out of work and is actively looking for work
536 blocked bool // m is blocked on a note
537 newSigstack bool // minit on C thread called sigaltstack
539 incgo bool // m is executing a cgo call
540 freeWait uint32 // if == 0, safe to free g0 and delete m (atomic)
544 ncgocall uint64 // number of cgo calls in total
545 ncgo int32 // number of cgo calls currently in progress
546 cgoCallersUse uint32 // if non-zero, cgoCallers in use temporarily
547 cgoCallers *cgoCallers // cgo traceback if crashing in cgo call
548 doesPark bool // non-P running threads: sysmon and newmHandoff never use .park
550 alllink *m // on allm
553 createstack [32]uintptr // stack that created this thread.
554 lockedExt uint32 // tracking for external LockOSThread
555 lockedInt uint32 // tracking for internal lockOSThread
556 nextwaitm muintptr // next m waiting for lock
557 waitunlockf func(*g, unsafe.Pointer) bool
558 waitlock unsafe.Pointer
563 freelink *m // on sched.freem
565 // mFixup is used to synchronize OS related m state
566 // (credentials etc) use mutex to access. To avoid deadlocks
567 // an atomic.Load() of used being zero in mDoFixupFn()
568 // guarantees fn is nil.
575 // these are here because they are too large to be on the stack
576 // of low-level NOSPLIT functions.
578 libcallpc uintptr // for cpu profiler
581 syscall libcall // stores syscall parameters on windows
583 vdsoSP uintptr // SP for traceback while in VDSO call (0 if not in call)
584 vdsoPC uintptr // PC for traceback while in VDSO call
586 // preemptGen counts the number of completed preemption
587 // signals. This is used to detect when a preemption is
588 // requested, but fails. Accessed atomically.
591 // Whether this is a pending preemption signal on this M.
592 // Accessed atomically.
599 // Up to 10 locks held by this m, maintained by the lock ranking code.
601 locksHeld [10]heldLockInfo
606 status uint32 // one of pidle/prunning/...
608 schedtick uint32 // incremented on every scheduler call
609 syscalltick uint32 // incremented on every system call
610 sysmontick sysmontick // last tick observed by sysmon
611 m muintptr // back-link to associated m (nil if idle)
616 deferpool []*_defer // pool of available defer structs (see panic.go)
617 deferpoolbuf [32]*_defer
619 // Cache of goroutine ids, amortizes accesses to runtime·sched.goidgen.
623 // Queue of runnable goroutines. Accessed without lock.
627 // runnext, if non-nil, is a runnable G that was ready'd by
628 // the current G and should be run next instead of what's in
629 // runq if there's time remaining in the running G's time
630 // slice. It will inherit the time left in the current time
631 // slice. If a set of goroutines is locked in a
632 // communicate-and-wait pattern, this schedules that set as a
633 // unit and eliminates the (potentially large) scheduling
634 // latency that otherwise arises from adding the ready'd
635 // goroutines to the end of the run queue.
637 // Note that while other P's may atomically CAS this to zero,
638 // only the owner P can CAS it to a valid G.
641 // Available G's (status == Gdead)
650 // Cache of mspan objects from the heap.
652 // We need an explicit length here because this field is used
653 // in allocation codepaths where write barriers are not allowed,
654 // and eliminating the write barrier/keeping it eliminated from
655 // slice updates is tricky, moreso than just managing the length
663 // traceSweep indicates the sweep events should be traced.
664 // This is used to defer the sweep start event until a span
665 // has actually been swept.
667 // traceSwept and traceReclaimed track the number of bytes
668 // swept and reclaimed by sweeping in the current sweep loop.
669 traceSwept, traceReclaimed uintptr
671 palloc persistentAlloc // per-P to avoid mutex
673 _ uint32 // Alignment for atomic fields below
675 // The when field of the first entry on the timer heap.
676 // This is updated using atomic functions.
677 // This is 0 if the timer heap is empty.
680 // The earliest known nextwhen field of a timer with
681 // timerModifiedEarlier status. Because the timer may have been
682 // modified again, there need not be any timer with this value.
683 // This is updated using atomic functions.
684 // This is 0 if there are no timerModifiedEarlier timers.
685 timerModifiedEarliest uint64
688 gcAssistTime int64 // Nanoseconds in assistAlloc
689 gcFractionalMarkTime int64 // Nanoseconds in fractional mark worker (atomic)
691 // gcMarkWorkerMode is the mode for the next mark worker to run in.
692 // That is, this is used to communicate with the worker goroutine
693 // selected for immediate execution by
694 // gcController.findRunnableGCWorker. When scheduling other goroutines,
695 // this field must be set to gcMarkWorkerNotWorker.
696 gcMarkWorkerMode gcMarkWorkerMode
697 // gcMarkWorkerStartTime is the nanotime() at which the most recent
698 // mark worker started.
699 gcMarkWorkerStartTime int64
701 // gcw is this P's GC work buffer cache. The work buffer is
702 // filled by write barriers, drained by mutator assists, and
703 // disposed on certain GC state transitions.
706 // wbBuf is this P's GC write barrier buffer.
708 // TODO: Consider caching this in the running G.
711 runSafePointFn uint32 // if 1, run sched.safePointFn at next safe point
713 // statsSeq is a counter indicating whether this P is currently
714 // writing any stats. Its value is even when not, odd when it is.
717 // Lock for timers. We normally access the timers while running
718 // on this P, but the scheduler can also do it from a different P.
721 // Actions to take at some time. This is used to implement the
722 // standard library's time package.
723 // Must hold timersLock to access.
726 // Number of timers in P's heap.
727 // Modified using atomic instructions.
730 // Number of timerDeleted timers in P's heap.
731 // Modified using atomic instructions.
734 // Race context used while executing timer functions.
737 // scannableStackSizeDelta accumulates the amount of stack space held by
738 // live goroutines (i.e. those eligible for stack scanning).
739 // Flushed to gcController.scannableStackSize once scannableStackSizeSlack
740 // or -scannableStackSizeSlack is reached.
741 scannableStackSizeDelta int64
743 // preempt is set to indicate that this P should be enter the
744 // scheduler ASAP (regardless of what G is running on it).
747 // Padding is no longer needed. False sharing is now not a worry because p is large enough
748 // that its size class is an integer multiple of the cache line size (for any of our architectures).
752 // accessed atomically. keep at top to ensure alignment on 32-bit systems.
754 lastpoll uint64 // time of last network poll, 0 if currently polling
755 pollUntil uint64 // time to which current poll is sleeping
759 // When increasing nmidle, nmidlelocked, nmsys, or nmfreed, be
760 // sure to call checkdead().
762 midle muintptr // idle m's waiting for work
763 nmidle int32 // number of idle m's waiting for work
764 nmidlelocked int32 // number of locked m's waiting for work
765 mnext int64 // number of m's that have been created and next M ID
766 maxmcount int32 // maximum number of m's allowed (or die)
767 nmsys int32 // number of system m's not counted for deadlock
768 nmfreed int64 // cumulative number of freed m's
770 ngsys uint32 // number of system goroutines; updated atomically
772 pidle puintptr // idle p's
774 nmspinning uint32 // See "Worker thread parking/unparking" comment in proc.go.
776 // Global runnable queue.
780 // disable controls selective disabling of the scheduler.
782 // Use schedEnableUser to control this.
784 // disable is protected by sched.lock.
786 // user disables scheduling of user goroutines.
788 runnable gQueue // pending runnable Gs
789 n int32 // length of runnable
792 // Global cache of dead G's.
795 stack gList // Gs with stacks
796 noStack gList // Gs without stacks
800 // Central cache of sudog structs.
804 // Central pool of available defer structs.
808 // freem is the list of m's waiting to be freed when their
809 // m.exited is set. Linked through m.freelink.
812 gcwaiting uint32 // gc is waiting to run
818 // While true, sysmon not ready for mFixup calls.
819 // Accessed atomically.
820 sysmonStarting uint32
822 // safepointFn should be called on each P at the next GC
823 // safepoint if p.runSafePointFn is set.
828 profilehz int32 // cpu profiling rate
830 procresizetime int64 // nanotime() of last change to gomaxprocs
831 totaltime int64 // ∫gomaxprocs dt up to procresizetime
833 // sysmonlock protects sysmon's actions on the runtime.
835 // Acquire and hold this mutex to block sysmon from interacting
836 // with the rest of the runtime.
839 _ uint32 // ensure timeToRun has 8-byte alignment
841 // timeToRun is a distribution of scheduling latencies, defined
842 // as the sum of time a G spends in the _Grunnable state before
843 // it transitions to _Grunning.
845 // timeToRun is protected by sched.lock.
846 timeToRun timeHistogram
849 // Values for the flags field of a sigTabT.
851 _SigNotify = 1 << iota // let signal.Notify have signal, even if from kernel
852 _SigKill // if signal.Notify doesn't take it, exit quietly
853 _SigThrow // if signal.Notify doesn't take it, exit loudly
854 _SigPanic // if the signal is from the kernel, panic
855 _SigDefault // if the signal isn't explicitly requested, don't monitor it
856 _SigGoExit // cause all runtime procs to exit (only used on Plan 9).
857 _SigSetStack // add SA_ONSTACK to libc handler
858 _SigUnblock // always unblock; see blockableSig
859 _SigIgn // _SIG_DFL action is to ignore the signal
862 // Layout of in-memory per-function information prepared by linker
863 // See https://golang.org/s/go12symtab.
864 // Keep in sync with linker (../cmd/link/internal/ld/pcln.go:/pclntab)
865 // and with package debug/gosym and with symtab.go in package runtime.
867 entryoff uint32 // start pc, as offset from moduledata.text/pcHeader.textStart
868 nameoff int32 // function name
870 args int32 // in/out args size
871 deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
877 cuOffset uint32 // runtime.cutab offset of this function's CU
878 funcID funcID // set for certain special runtime functions
881 nfuncdata uint8 // must be last, must end on a uint32-aligned boundary
884 // Pseudo-Func that is returned for PCs that occur in inlined code.
885 // A *Func can be either a *_func or a *funcinl, and they are distinguished
886 // by the first uintptr.
887 type funcinl struct {
888 ones uint32 // set to ^0 to distinguish from _func
889 entry uintptr // entry of the real (the "outermost") frame
895 // layout of Itab known to compilers
896 // allocated in non-garbage-collected memory
897 // Needs to be in sync with
898 // ../cmd/compile/internal/reflectdata/reflect.go:/^func.WriteTabs.
902 hash uint32 // copy of _type.hash. Used for type switches.
904 fun [1]uintptr // variable sized. fun[0]==0 means _type does not implement inter.
907 // Lock-free stack node.
908 // Also known to export_test.go.
914 type forcegcstate struct {
920 // extendRandom extends the random numbers in r[:n] to the whole slice r.
921 // Treats n<0 as n==0.
922 func extendRandom(r []byte, n int) {
927 // Extend random bits using hash function & time seed
932 h := memhash(unsafe.Pointer(&r[n-w]), uintptr(nanotime()), uintptr(w))
933 for i := 0; i < goarch.PtrSize && n < len(r); i++ {
941 // A _defer holds an entry on the list of deferred calls.
942 // If you add a field here, add code to clear it in freedefer and deferProcStack
943 // This struct must match the code in cmd/compile/internal/ssagen/ssa.go:deferstruct
944 // and cmd/compile/internal/ssagen/ssa.go:(*state).call.
945 // Some defers will be allocated on the stack and some on the heap.
946 // All defers are logically part of the stack, so write barriers to
947 // initialize them are not required. All defers must be manually scanned,
948 // and for heap defers, marked.
952 // openDefer indicates that this _defer is for a frame with open-coded
953 // defers. We have only one defer record for the entire frame (which may
954 // currently have 0, 1, or more defers active).
956 sp uintptr // sp at time of defer
957 pc uintptr // pc at time of defer
958 fn func() // can be nil for open-coded defers
959 _panic *_panic // panic that is running defer
960 link *_defer // next defer on G; can point to either heap or stack!
962 // If openDefer is true, the fields below record values about the stack
963 // frame and associated function that has the open-coded defer(s). sp
964 // above will be the sp for the frame, and pc will be address of the
965 // deferreturn call in the function.
966 fd unsafe.Pointer // funcdata for the function associated with the frame
967 varp uintptr // value of varp for the stack frame
968 // framepc is the current pc associated with the stack frame. Together,
969 // with sp above (which is the sp associated with the stack frame),
970 // framepc/sp can be used as pc/sp pair to continue a stack trace via
975 // A _panic holds information about an active panic.
977 // A _panic value must only ever live on the stack.
979 // The argp and link fields are stack pointers, but don't need special
980 // handling during stack growth: because they are pointer-typed and
981 // _panic values only live on the stack, regular stack pointer
982 // adjustment takes care of them.
984 argp unsafe.Pointer // pointer to arguments of deferred call run during panic; cannot move - known to liblink
985 arg interface{} // argument to panic
986 link *_panic // link to earlier panic
987 pc uintptr // where to return to in runtime if this panic is bypassed
988 sp unsafe.Pointer // where to return to in runtime if this panic is bypassed
989 recovered bool // whether this panic is over
990 aborted bool // the panic was aborted
995 type stkframe struct {
996 fn funcInfo // function being run
997 pc uintptr // program counter within fn
998 continpc uintptr // program counter where execution can continue, or 0 if not
999 lr uintptr // program counter at caller aka link register
1000 sp uintptr // stack pointer at pc
1001 fp uintptr // stack pointer at caller aka frame pointer
1002 varp uintptr // top of local variables
1003 argp uintptr // pointer to function arguments
1004 arglen uintptr // number of bytes at argp
1005 argmap *bitvector // force use of this argmap
1008 // ancestorInfo records details of where a goroutine was started.
1009 type ancestorInfo struct {
1010 pcs []uintptr // pcs from the stack of this goroutine
1011 goid int64 // goroutine id of this goroutine; original goroutine possibly dead
1012 gopc uintptr // pc of go statement that created this goroutine
1016 _TraceRuntimeFrames = 1 << iota // include frames for internal runtime functions.
1017 _TraceTrap // the initial PC, SP are from a trap, not a return PC from a call
1018 _TraceJumpStack // if traceback is on a systemstack, resume trace at g that called into it
1021 // The maximum number of frames we print for a traceback
1022 const _TracebackMaxFrames = 100
1024 // A waitReason explains why a goroutine has been stopped.
1025 // See gopark. Do not re-use waitReasons, add new ones.
1026 type waitReason uint8
1029 waitReasonZero waitReason = iota // ""
1030 waitReasonGCAssistMarking // "GC assist marking"
1031 waitReasonIOWait // "IO wait"
1032 waitReasonChanReceiveNilChan // "chan receive (nil chan)"
1033 waitReasonChanSendNilChan // "chan send (nil chan)"
1034 waitReasonDumpingHeap // "dumping heap"
1035 waitReasonGarbageCollection // "garbage collection"
1036 waitReasonGarbageCollectionScan // "garbage collection scan"
1037 waitReasonPanicWait // "panicwait"
1038 waitReasonSelect // "select"
1039 waitReasonSelectNoCases // "select (no cases)"
1040 waitReasonGCAssistWait // "GC assist wait"
1041 waitReasonGCSweepWait // "GC sweep wait"
1042 waitReasonGCScavengeWait // "GC scavenge wait"
1043 waitReasonChanReceive // "chan receive"
1044 waitReasonChanSend // "chan send"
1045 waitReasonFinalizerWait // "finalizer wait"
1046 waitReasonForceGCIdle // "force gc (idle)"
1047 waitReasonSemacquire // "semacquire"
1048 waitReasonSleep // "sleep"
1049 waitReasonSyncCondWait // "sync.Cond.Wait"
1050 waitReasonTimerGoroutineIdle // "timer goroutine (idle)"
1051 waitReasonTraceReaderBlocked // "trace reader (blocked)"
1052 waitReasonWaitForGCCycle // "wait for GC cycle"
1053 waitReasonGCWorkerIdle // "GC worker (idle)"
1054 waitReasonPreempted // "preempted"
1055 waitReasonDebugCall // "debug call"
1058 var waitReasonStrings = [...]string{
1060 waitReasonGCAssistMarking: "GC assist marking",
1061 waitReasonIOWait: "IO wait",
1062 waitReasonChanReceiveNilChan: "chan receive (nil chan)",
1063 waitReasonChanSendNilChan: "chan send (nil chan)",
1064 waitReasonDumpingHeap: "dumping heap",
1065 waitReasonGarbageCollection: "garbage collection",
1066 waitReasonGarbageCollectionScan: "garbage collection scan",
1067 waitReasonPanicWait: "panicwait",
1068 waitReasonSelect: "select",
1069 waitReasonSelectNoCases: "select (no cases)",
1070 waitReasonGCAssistWait: "GC assist wait",
1071 waitReasonGCSweepWait: "GC sweep wait",
1072 waitReasonGCScavengeWait: "GC scavenge wait",
1073 waitReasonChanReceive: "chan receive",
1074 waitReasonChanSend: "chan send",
1075 waitReasonFinalizerWait: "finalizer wait",
1076 waitReasonForceGCIdle: "force gc (idle)",
1077 waitReasonSemacquire: "semacquire",
1078 waitReasonSleep: "sleep",
1079 waitReasonSyncCondWait: "sync.Cond.Wait",
1080 waitReasonTimerGoroutineIdle: "timer goroutine (idle)",
1081 waitReasonTraceReaderBlocked: "trace reader (blocked)",
1082 waitReasonWaitForGCCycle: "wait for GC cycle",
1083 waitReasonGCWorkerIdle: "GC worker (idle)",
1084 waitReasonPreempted: "preempted",
1085 waitReasonDebugCall: "debug call",
1088 func (w waitReason) String() string {
1089 if w < 0 || w >= waitReason(len(waitReasonStrings)) {
1090 return "unknown wait reason"
1092 return waitReasonStrings[w]
1099 forcegc forcegcstate
1103 // allpLock protects P-less reads and size changes of allp, idlepMask,
1104 // and timerpMask, and all writes to allp.
1106 // len(allp) == gomaxprocs; may change at safe points, otherwise
1109 // Bitmask of Ps in _Pidle list, one bit per P. Reads and writes must
1110 // be atomic. Length may change at safe points.
1112 // Each P must update only its own bit. In order to maintain
1113 // consistency, a P going idle must the idle mask simultaneously with
1114 // updates to the idle P list under the sched.lock, otherwise a racing
1115 // pidleget may clear the mask before pidleput sets the mask,
1116 // corrupting the bitmap.
1118 // N.B., procresize takes ownership of all Ps in stopTheWorldWithSema.
1120 // Bitmask of Ps that may have a timer, one bit per P. Reads and writes
1121 // must be atomic. Length may change at safe points.
1124 // Pool of GC parked background workers. Entries are type
1125 // *gcBgMarkWorkerNode.
1126 gcBgMarkWorkerPool lfstack
1128 // Total number of gcBgMarkWorker goroutines. Protected by worldsema.
1129 gcBgMarkWorkerCount int32
1131 // Information about what cpu features are available.
1132 // Packages outside the runtime should not use these
1133 // as they are not an external api.
1134 // Set on startup in asm_{386,amd64}.s
1135 processorVersionInfo uint32
1138 goarm uint8 // set by cmd/link on arm systems
1141 // Set by the linker so the runtime can determine the buildmode.
1143 islibrary bool // -buildmode=c-shared
1144 isarchive bool // -buildmode=c-archive
1147 // Must agree with internal/buildcfg.Experiment.FramePointer.
1148 const framepointer_enabled = GOARCH == "amd64" || GOARCH == "arm64"