See the AppScope repo to view all issues.
2023-07-19 - Update to Feature Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.4.1
x86
: https://cdn.cribl.io/dl/scope/1.4.1/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.4.1/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.4.1/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.4.1/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.4.1
.
Cribl's AppScope Helm Chart is available here: https://github.com/criblio/helm-charts/tree/master/helm-chart-sources/appscope
Beginning in version 1.4.1, AppScope applies Rules to new processes in existing containers. That is, within any container that exists when you start scoping by Rule, AppScope will automatically begin monitoring any new process that matches the Rule.
This holds true whether you are using the scope rules
CLI command, or the AppScope Rules settings in the AppScope Source UI in Cribl Edge or Cribl Stream. Related issue: 1523.
2023-07-19 - Feature Release
This release has been replaced by AppScope 1.4.1. Therefore, the AppScope recommends against downloading AppScope 1.4.0 assets from Docker or the Cribl CDN.
AppScope 1.4.0 integrates more deeply with Cribl Edge:
The AppScope CLI has new functionality to support the Cribl Edge integration, including:
rules
command that specifies processes to scope, along with configs to apply to those processes.start
, stop
, attach
, detach
, update
, and inspect
commands.In general, AppScope 1.4.0 expands support for Kubernetes.
unknown terminal type
error.scope ps
no longer fails to shows child processes.LD_PRELOAD
mechanism.
and the scoped application performs a fork and exec, AppScope no longer crashes the application.2023-06-14 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.3.4
x86
: https://cdn.cribl.io/dl/scope/1.3.4/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.3.4/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.3.4/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.3.4/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.3.4
.
backtrace
symbol. Before this fix, certain scoped applications would segfault when they called backtrace
.label
section.2023-05-17 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.3.3
x86
: https://cdn.cribl.io/dl/scope/1.3.3/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.3.3/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.3.3/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.3.3/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.3.3
.
AppScope 1.3.3 removes our support for two Go versions that the Go project no longer supports: 1.9 and 1.10. This makes the AppScope code less complicated and more robust. Related issue: #1452.
1461 In libc musl environments, attaching to an already-running but idle process now works normally. (Idle means not producing events at that moment.)
scope ps
, scope inspect
, scope update
) would not work, because AppScope's periodic thread would not start. For example, attaching AppScope to the Nginx master process running in an Alpine container, you would be unable to use the CLI to interact with the process. (This problem existed only in libc musl environments.)2023-04-19 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.3.2
x86
: https://cdn.cribl.io/dl/scope/1.3.2/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.3.2/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.3.2/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.3.2/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.3.2
.
AppScope 1.3.2 introduces support for Go version 1.20.
scope k8s
command now preloads the application image into the k8s cluster, avoiding failed to call webhook
errors.scope k8s
command now calls an up-to-date version of a k8s library needed for obtaining signed certificates from the k8s Certificate Authority. This fixes a problem where running in newer versions of k8s produced the server doesn't have a resource type "certificatesigningrequests"
errors. scope run
command now works as expected, and no longer encounters secure_getenv: symbol not found
errors.pcre2
functions internally is improved, fixing various problems including Go crashes under certain conditions.2023-03-21 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.3.1
x86
: https://cdn.cribl.io/dl/scope/1.3.1/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.3.1/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.3.1/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.3.1/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.3.1
.
2023-03-21 - Feature Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.3.0
x86
: https://cdn.cribl.io/dl/scope/1.3.0/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.3.0/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.3.0/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.3.0/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.3.0
.
AppScope 1.3.0 introduces features that support analyzing crashes, obtaining snapshots of processes, troubleshooting transports, and dynamically managing configs. Meanwhile, AppScope's architecture, connection and payload handling, container-awareness, and CLI are all improved.
AppScope 1.3.0 introduces support for:
Whenever a scoped app crashes, AppScope can obtain a core dump, a backtrace (i.e., stack trace), or both, while capturing supplemental information in text files.
AppScope can generate a snapshot file containing debug information about processes that are running normally or crashing, unscoped or scoped.
AppScope now uses inter-process communication (IPC) to interact with processes in new ways. These include determining whether the process is scoped or not. If the process is scoped, AppScope can:
AppScope now uses a backoff algorithm for connections to avoid creating excessive network traffic and log entries. When a remote destination that AppScope tries to connect to rejects the connection or is not available, AppScope retries the connection at a progressively slower rate.
When the payloads feature is enabled, setting SCOPE_PAYLOAD_TO_DISK
to true
now guarantees that AppScope will write payloads to the local directory specified in SCOPE_PAYLOAD_DIR
.
The ldscope
utility no longer exists, and you can use CLI commands instead; ldscope.log
has been renamed as libscope.log
.
The AppScope CLI is enhanced in the following ways:
scope start
can now attach to processes running in rootless and nested containers.scope detach
, when run with the new --all
flag, detaches from all processes at once.scope stop
, a new command, runs scope detach --all
, removes the filter file from the system, and removes scope
from service configurations. This undoes the effects of the scope attach
, scope start
, and/or scope service
commands.scope snapshot
obtains debug information about a running or crashing process, regardless of whether or not the process is scoped.scope --passthrough
replaces scope run --passthrough
.Three commands use IPC, which is new in AppScope 1.3.0. scope inspect
and scope update
are completely new, while scope ps
has new capabilities thanks to IPC.
scope inspect
retrieves the AppScope config currently in effect and determines the status of the transport AppScope is trying to use.scope update
modifies the current AppScope config.scope ps
now determines whether the processes it lists are scoped or not.scope terraform plan
– no longer causes Terraform to crash. scope events -f
to see the scoped app's events scrolling – works correctly again, fixing a regression in recent versions of AppScope.GET
or SET
command.scope ps
command no longer erroneously reports that a process is scoped even after scope detach
has been run for that process.2023-01-18 - Maintenance Release
AppScope 1.2.2 fixes a critical security vulnerability in OpenSSL: CVE-2022-3602. Cribl strongly recommends upgrading to AppScope 1.2.2 as soon as possible. See this AppScope security advisory.
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.2.2
x86
: https://cdn.cribl.io/dl/scope/1.2.2/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.2.2/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.2.2/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.2.2/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.2.2
.
AppScope 1.2.2 introduces:
SCOPE_PAYLOAD_TO_DISK
together with existing env vars SCOPE_CRIBL_ENABLE
and SCOPE_PAYLOAD_ENABLE
, as described here. See issue #1158.AppScope 1.2.2 also updates the UPX executable packer from version 4.0.0 to version 4.0.1. See issue #1214.
2022-12-07 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.2.1
x86
: https://cdn.cribl.io/dl/scope/1.2.1/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.2.1/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.2.1/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.2.1/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.2.1
.
AppScope 1.2.1 introduces:
--all
or -a
flag for the scope detach
command. Running scope detach --all
detaches AppScope from all processes.failed to find libc in target process
error.mysqld
to crash.2022-11-09 - Feature Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.2.0
x86
: https://cdn.cribl.io/dl/scope/1.2.0/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.2.0/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.2.0/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.2.0/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.2.0
.
AppScope 1.2.0 introduces substantial new functionality:
scope start
command, which takes a filter list as an argument. The filter list has an allowlist section for specifying which processes to scope (along with desired configs), and an optional denylist section for specifying which processes not to scope. Related issues: #987,#1027,#1038,#1069.scope.yml
) and storing them in an AppScope Config Library. scope detach
command, which stops scoping a process without terminating that process. Related issues: #370,#1039,#1085,#1087,#1088,#1089,#1094,#1107,#1129.This release also improves the scope ps
command, which now lists all scoped processes rather than listing all processes into which the library was loaded. Related issue: #1097.
2022-09-06 - Maintenance Release
Assets are available from the Cribl CDN at the links below.
AppScope for x86
: https://cdn.cribl.io/dl/scope/1.1.3/linux/x86_64/scopeAppScope for ARM
: https://cdn.cribl.io/dl/scope/1.1.3/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.1.3/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.1.3/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.1.3
.
scope attach
and then select an ID that's beyond the range of the resulting list, AppScope now handles this gracefully, and the AppScope CLI no longer crashes.2022-08-09 - Maintenance Release
Assets are available from the Cribl CDN at the links below.
AppScope for x86
: https://cdn.cribl.io/dl/scope/1.1.2/linux/x86_64/scopeAppScope for ARM
: https://cdn.cribl.io/dl/scope/1.1.2/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.1.2/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.1.2/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.1.2
.
#1025 The process‑start message (the start.msg event) now includes a total of four identifiers. By itself, the new UUID process ID is unique for a given machine. In principle, UUID together with the new Machine ID constitutes a tuple ID that is unique across all machine namespaces. Here's a summary of the IDs available in AppScope 1.1.2:
uuid
and a value in canonical UUID form. UUID is a universally-unique process identifier, meaning that no two processes will ever have the same UUID value on the same machine.machine_id
and a value that AppScope obtains from /etc/machine-id
. The machine ID uniquely identifies the host, as described in the man page. When /etc/machine-id
is not available (e.g., in Alpine, or in a container), AppScope generates the machine ID using a repeatable MD5 hash of the host's first MAC address. Two containers on the same host can have the same machine ID.pid
and a value that is always unique at any given time, but that the machine can reuse for different processes at different times.id
and a value that concatenates (and may truncate) the scoped app's hostname, procname, and command. This value is not guaranteed to be unique.scope service
on Linux distros that use the OpenRC init manager (e.g., Alpine), as well as on distros that use systemd (which was already supported). See the CLI reference.scope service
command no longer ignores the -c
(--cribldest
), -e
(--eventdest
), and -m
(--metricdest
) options.2022-07-12 - Maintenance Release
Assets are available from the Cribl CDN at the links below.
AppScope for x86
: https://cdn.cribl.io/dl/scope/1.1.1/linux/x86_64/scopeAppScope for ARM
: https://cdn.cribl.io/dl/scope/1.1.1/linux/aarch64/scopeAWS Lambda Layer for x86
: https://cdn.cribl.io/dl/scope/1.1.1/linux/x86_64/aws-lambda-layer.zipAWS Lambda Layer for ARM
: https://cdn.cribl.io/dl/scope/1.1.1/linux/aarch64/aws-lambda-layer.zipTo obtain the MD5 checksum for any file above, add .md5
to the file path.
Assets other than AWS Lambda Layers are available in the Docker container tagged cribl/scope:1.1.1
.
execve
and execv
system calls, and overrides some of the sandboxing that sshd normally imposes using setrlimit
. Interposing execv
is new, and gives AppScope visibility into sshd child processes. Changing setrlimit
settings enables AppScope to perform actions required by AppScope's configured backend and transport, such as establishing connections, creating threads, and creating files.2022-06-28 - Minor Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.1.0
x86
: https://cdn.cribl.io/dl/scope/1.1.0/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.1.0/linux/aarch64/scopeAppScope 1.1.0 introduces improved capabilities for scoping Go applications:
Usability improvements include:
start.msg
) event which AppScope sends when log level is info
or debug
is now documented in the schema.SCOPE_ERROR_SIGNAL_HANDLER
environment variable, provided for situations where a scoped app is crashing. Setting this variable to true
sends backtrace information to the log, which can help you diagnose problems. SCOPE_ALLOW_BINARY_CONSOLE
can now be set using the environment variable (prior to 1.1.0, the only available method) or in the config file. For the library, its default value has been changed from false
to true
. Meanwhile, the CLI still defaults to allowbinary=false
, because allowing binary data in CLI output makes sense only in rare cases.cribl
section describing Cribl backend configuration.2022-05-10 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.0.4
x86
: https://cdn.cribl.io/dl/scope/1.0.4/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.0.4/linux/aarch64/scopeAppScope 1.0.4 aims to resolve reported connectivity issues.
2022-04-12 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.0.3
x86
: https://cdn.cribl.io/dl/scope/1.0.3/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.0.3/linux/aarch64/scopeAppScope 1.0.3 improves the quality of results when scoping Go executables:
unlinkat
and getdents
. AppScope already had the ability to interpose the equivalent system calls in C (unlink
, unlinkat
, readdir
).openat
function, AppScope no longer fails to collect fs.open
events. This brings AppScope up to date with changes to openat
in recent versions of Go.2022-03-15 - Maintenance Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.0.2
x86
: https://cdn.cribl.io/dl/scope/1.0.2/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.0.2/linux/aarch64/scopeAppScope 1.0.2 introduces fine-grained control of scope events
output:
#826 When you do filtering or matching with scope events
, AppScope now shows all results by default. (The filtering/matching options are --sort
, --match
, --source
, and --sourcetype
.) Alternatively, you can limit the number of events with the --last
option, e.g., scope events --last 20
.
scope events
shows the last 20 results by default. Use --last
to specify a different limit, or --all
to see all results.scope events --fields
, and the output will show only those field names and their values alongside each event. (If you want to restrict output to only those events which contain the selected fields, add the --match
option.)--sort
option to sort by a top-level field (i.e., a top-level element of the event body). By default, this returns events sorted in descending order, both for numeric values (such as timestamps) and string values (such as the values for the proc
field). To sort in ascending order, add the --reverse
flag.Another usability improvement applies to the CLI as a whole:
scope
now responds with an error.scope events
is now rendered on one line._chk
) functions to libscope.so
, without which a scoped command could fail when that command had been compiled with a particular GCC toolchain.http_method
field in HTTP events contained junk. This work continues in #829.2022-02-28 - Update to GA Release
Assets are available via Docker and the Cribl CDN at the links below.
Docker
: cribl/scope:1.0.1
x86
: https://cdn.cribl.io/dl/scope/1.0.1/linux/x86_64/scopeARM
: https://cdn.cribl.io/dl/scope/1.0.1/linux/aarch64/scopeAppScope 1.0.1 updates event and metric names. The AppScope team strongly recommends using AppScope 1.0.1 and not 1.0.0.
2022-02-15 - GA Release
This release has been replaced by AppScope 1.0.1.
AppScope's official v1 release includes advances in metric capture and forwarding, stabilization, formal schema definition and associated standardization, and enhanced filesystem events.
missing uv_fileno
error is reduced to debug level.fs
events.fs
events.net.open
and net.close
metrics.cribl
connection header.edge
transport type, with its own default search path and CLI support.unix://
destination in the CLI.apt-get
no longer hangs.2021-12-21 - Maintenance Pre-Release
[2021-10-20 17:48:36.974]
. This timestamp format is ambiguous, and a fix is planned for AppScope 1.0.0. See #678. proc.start
metric and process start event (also known as TCP JSON connection header).2021-10-26 - Maintenance Pre-Release
libscope.so
library, not LogStream as was the case previously.scope run -u scope.yml -- foo
where scope.yml
is the configuration file and foo
is the command being scoped.scope attach
are now resolved and no longer apply.2021-10-05 - Maintenance Pre-Release
custom
section in scope.yml
. scope_protocol.yml
and merging its content into scope.yml
.tags
section of scope.yml
to top level, removing it from the metric
> format
section. This change is not backwards-compatible. 2021-09-14 - Maintenance Pre-Release
scope k8s
now (1) supports specifying the namespace to install into, and (2) when the ConfigMap in that namespace is edited, automatically propagates the changes to all namespaces instrumented with the label scope=enabled
.2021-09-01 - Maintenance/Hotfix Pre-Release
2021-08-17 - Maintenance Pre-Release
scope
attaches to a bash process, it now emits events for any child processes.SCOPE_CONNECT_TIMEOUT_SECS
, to configure wait behavior. This prevents events and metrics from being dropped when a scoped command exits quickly while using a connection with network latency.scope prune
now rejects negative numbers as arguments. Positive arguments implicitly mean "how many sessions back" to delete or keep.net
event type, which prevents them from being dropped by periodic threads.This pre-release addresses the following issues:
2021-07-20 - Maintenance Pre-Release
This pre-release addresses the following issues:
scope ps
command to report on all processes into which the libscope library is loaded.scope attach NAME
.authToken
to pass to LogStream as a header, using scope run -a ${authToken}
in the CLI, or the cribl
section of scope.yml
.-n
or --nobreaker
option for configuring LogStream breaker behavior from AppScope. This option prevents LogStream from running Event Breakers on an AppScope Source. The --nobreaker
option can only be used with LogStream starting with version 3.0.4.WARNING: Session history will be stored in $HOME/.scope/history and owned by root
.master
and release/*
branches.website/
content.Improvement: #350 In docker-build
, add support for:
build
command, for example: make docker-build CMD="make coreall"
.-u $(id -u):$(id -g)
to Docker so that the current user owns build results..dockerignore
to omit unnecessary and potentially large items like **/core
, website/
, test/testContainers
.scope flows
command now works when stdin
is a pipe._
) rather than periods (.
). The http.status
and http.target
dimensions have been corrected. This change is not backwards-compatible.http.client.duration
and http.server.duration
are now the correct type, timer
. This change is not backwards-compatible.-d
and -k
options to scope prune
.2021-07-02 - Maintenance Pre-Release
This pre-release addresses the following issues:
ldscope --help configuration | grep TLS
.scope logs
command to view logs from the CLI.scope watch
command to run AppScope at intervals of seconds, minutes, or hours, from the CLI.git clone
to crash.2021-04-26 - Maintenance Pre-Release
This pre-release addresses the following issues:
‑buildmode=pie
.2021-04-01 - Maintenance Pre-Release
This pre-release addresses the following issues:
scope run
flags to facilitate sending scope data to third-party systems; also add a scope k8s
command to facilitate installing a mutating admission webhook in a Kubernetes environment.2021-02-05 - Maintenance Pre-Release
This pre-release addresses the following issues:
2021-02-05 - Initial Pre-Release
AppScope is now a thing. Its public repo is at https://github.com/criblio/appscope.