Merge remote-tracking branch 'upstream/master'
This commit is contained in:
Alin Sinpalean
+5
-3
@@ -7,11 +7,13 @@
|
||||
- UX changes to nav & side menu
|
||||
- New dashboard grid layout system
|
||||
|
||||
# 4.5.0 (unreleased)
|
||||
|
||||
## Enhancements
|
||||
# 4.5.0 (2017-09-14)
|
||||
|
||||
## Fixes & Enhancements since beta1
|
||||
* **Security**: Security fix for api vulnerability (in multiple org setups).
|
||||
* **Shortcuts**: Adds shortcut for creating new dashboard [#8876](https://github.com/grafana/grafana/pull/8876) thx [@mtanda](https://github.com/mtanda)
|
||||
* **Graph**: Right Y-Axis label position fixed [#9172](https://github.com/grafana/grafana/pull/9172)
|
||||
* **General**: Improve rounding of time intervals [#9197](https://github.com/grafana/grafana/pull/9197), thx [@alin-amana](https://github.com/alin-amana)
|
||||
|
||||
# 4.5.0-beta1 (2017-09-05)
|
||||
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
+++
|
||||
title = "Internal metrics"
|
||||
description = "Internal metrics exposed by Grafana"
|
||||
keywords = ["grafana", "metrics", "internal metrics"]
|
||||
type = "docs"
|
||||
[menu.docs]
|
||||
parent = "admin"
|
||||
weight = 8
|
||||
+++
|
||||
|
||||
# Internal metrics
|
||||
|
||||
Grafana collects some metrics about it self internally. Currently Grafana supports pushing metrics to graphite and exposing them to be scraped by Prometheus.
|
||||
|
||||
To enabled internal metrics you have to enable it under the [metrics] section in your [grafana.ini](http://docs.grafana.org/installation/configuration/#enabled-6) config file.If you want to push metrics to graphite you have also have to configure the [metrics.graphite](http://docs.grafana.org/installation/configuration/#metrics-graphite) section.
|
||||
@@ -41,7 +41,7 @@ Proxy access means that the Grafana backend will proxy all requests from the bro
|
||||
Click the ``Select metric`` link to start navigating the metric space. One you start you can continue using the mouse
|
||||
or keyboard arrow keys. You can select a wildcard and still continue.
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_query1_still.png" class="docs-image--center"
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_query1_still.png"
|
||||
animated-gif="/img/docs/v45/graphite_query1.gif" >}}
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ a function is selected it will be added and your focus will be in the text box o
|
||||
a parameter just click on it and it will turn into a text box. To delete a function click the function name followed
|
||||
by the x icon.
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_query2_still.png" class="docs-image--center"
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_query2_still.png"
|
||||
animated-gif="/img/docs/v45/graphite_query2.gif" >}}
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@ by the x icon.
|
||||
|
||||
Some functions like aliasByNode support an optional second argument. To add this parameter specify for example 3,-2 as the first parameter and the function editor will adapt and move the -2 to a second parameter. To remove the second optional parameter just click on it and leave it blank and the editor will remove it.
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_query3_still.png" class="docs-image--center"
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_query3_still.png"
|
||||
animated-gif="/img/docs/v45/graphite_query3.gif" >}}
|
||||
|
||||
|
||||
@@ -68,6 +68,10 @@ Some functions like aliasByNode support an optional second argument. To add this
|
||||
|
||||
You can reference queries by the row “letter” that they’re on (similar to Microsoft Excel). If you add a second query to a graph, you can reference the first query simply by typing in #A. This provides an easy and convenient way to build compounded queries.
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/graphite_nested_queries_still.png"
|
||||
animated-gif="/img/docs/v45/graphite_nested_queries.gif" >}}
|
||||
|
||||
|
||||
## Point consolidation
|
||||
|
||||
All Graphite metrics are consolidated so that Graphite doesn't return more data points than there are pixels in the graph. By default,
|
||||
|
||||
@@ -41,9 +41,7 @@ mode is also more secure as the username & password will never reach the browser
|
||||
|
||||
## Query Editor
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/influxdb_query_still.png" class="docs-image--center"
|
||||
animated-gif="/img/docs/v45/influxdb_query.gif" >}}
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/influxdb_query_still.png" class="docs-image--no-shadow" animated-gif="/img/docs/v45/influxdb_query.gif" >}}
|
||||
|
||||
You find the InfluxDB editor in the metrics tab in Graph or Singlestat panel's edit mode. You enter edit mode by clicking the
|
||||
panel title, then edit. The editor allows you to select metrics and tags.
|
||||
@@ -59,10 +57,8 @@ will automatically adjust the filter tag condition to use the InfluxDB regex mat
|
||||
|
||||
### Field & Aggregation functions
|
||||
In the `SELECT` row you can specify what fields and functions you want to use. If you have a
|
||||
group by time you need an aggregation function. Some functions like derivative require an aggregation function.
|
||||
|
||||
The editor tries simplify and unify this part of the query. For example:
|
||||
|
||||
group by time you need an aggregation function. Some functions like derivative require an aggregation function. The editor tries simplify and unify this part of the query. For example:<br>
|
||||
<br>
|
||||
|
||||
The above will generate the following InfluxDB `SELECT` clause:
|
||||
|
||||
|
||||
@@ -11,8 +11,7 @@ weight = 7
|
||||
|
||||
# Using MySQL in Grafana
|
||||
|
||||
> Only available in Grafana v4.3+. This data source is not ready for
|
||||
> production use, currently in development (alpha state).
|
||||
> Only available in Grafana v4.3+.
|
||||
|
||||
Grafana ships with a built-in MySQL data source plugin that allow you to query any visualize
|
||||
data from a MySQL compatible database.
|
||||
@@ -58,8 +57,7 @@ If the `Format as` query option is set to `Table` then you can basically do any
|
||||
|
||||
Query editor with example query:
|
||||
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/mysql_table_query.png" >}}
|
||||
|
||||
The query:
|
||||
|
||||
|
||||
@@ -39,7 +39,8 @@ Name | Description
|
||||
|
||||
Open a graph in edit mode by click the title > Edit (or by pressing `e` key while hovering over panel).
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/prometheus_query_editor_still.png"
|
||||
animated-gif="/img/docs/v45/prometheus_query_editor.gif" >}}
|
||||
|
||||
Name | Description
|
||||
------- | --------
|
||||
|
||||
@@ -12,7 +12,7 @@ weight = 2
|
||||
|
||||
# Table Panel
|
||||
|
||||
<img src="/assets/img/features/table-panel.png">
|
||||
<img class="screenshot" src="/assets/img/features/table-panel.png">
|
||||
|
||||
The new table panel is very flexible, supporting both multiple modes for time series as well as for
|
||||
table, annotation and raw JSON data. It also provides date formatting and value formatting and coloring options.
|
||||
@@ -22,55 +22,63 @@ To view table panels in action and test different configurations with sample dat
|
||||
## Options overview
|
||||
|
||||
The table panel has many ways to manipulate your data for optimal presentation.
|
||||
{{< docs-imagebox img="/img/docs/v45/table_options.png" class="docs-image--no-shadow" max-width= "500px" >}}
|
||||
|
||||
<img class="no-shadow" src="/img/docs/v2/table-config2.png">
|
||||
|
||||
1. `Data`: Control how your query is transformed into a table.
|
||||
2. `Table Display`: Table display options.
|
||||
3. `Column Styles`: Column value formatting and display options.
|
||||
2. `Paging`: Table display options.
|
||||
|
||||
|
||||
## Data to Table
|
||||
|
||||
<img class="no-shadow" src="/img/docs/v2/table-data-options.png">
|
||||
{{< docs-imagebox img="/img/docs/v45/table_data_options.png" max-width="500px" class="docs-image--right">}}
|
||||
|
||||
The data section contains the **To Table Transform (1)**. This is the primary option for how your data/metric
|
||||
query should be transformed into a table format. The **Columns (2)** option allows you to select what columns
|
||||
you want in the table. Only applicable for some transforms.
|
||||
|
||||
<div class="clearfix"></div>
|
||||
|
||||
### Time series to rows
|
||||
|
||||
<img src="/img/docs/v2/table_ts_to_rows2.png">
|
||||
{{< docs-imagebox img="/img/docs/v45/table_ts_to_rows.png" >}}
|
||||
|
||||
In the most simple mode you can turn time series to rows. This means you get a `Time`, `Metric` and a `Value` column. Where `Metric` is the name of the time series.
|
||||
|
||||
### Time series to columns
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/table_ts_to_columns.png" >}}
|
||||
|
||||
|
||||
This transform allows you to take multiple time series and group them by time. Which will result in the primary column being `Time` and a column for each time series.
|
||||
|
||||
### Time series aggregations
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/table_ts_to_aggregations.png" >}}
|
||||
|
||||
This table transformation will lay out your table into rows by metric, allowing columns of `Avg`, `Min`, `Max`, `Total`, `Current` and `Count`. More than one column can be added.
|
||||
|
||||
### Annotations
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/table_annotations.png" >}}
|
||||
|
||||
|
||||
If you have annotations enabled in the dashboard you can have the table show them. If you configure this
|
||||
mode then any queries you have in the metrics tab will be ignored.
|
||||
|
||||
### JSON Data
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/table_json_data.png" max-width="500px" >}}
|
||||
|
||||
If you have an Elasticsearch **Raw Document** query or an Elasticsearch query without a `date histogram` use this
|
||||
transform mode and pick the columns using the **Columns** section.
|
||||
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/elastic_raw_doc.png" >}}
|
||||
|
||||
## Table Display
|
||||
|
||||
<img class="no-shadow" src="/img/docs/v2/table-display.png">
|
||||
{{< docs-imagebox img="/img/docs/v45/table_paging.png" class="docs-image--no-shadow docs-image--right" max-width="350px" >}}
|
||||
|
||||
1. `Pagination (Page Size)`: The table display fields allow you to control The `Pagination` (page size) is the threshold at which the table rows will be broken into pages. For example, if your table had 95 records with a pagination value of 10, your table would be split across 9 pages.
|
||||
2. `Scroll`: The `scroll bar` checkbox toggles the ability to scroll within the panel, when unchecked, the panel height will grow to display all rows.
|
||||
@@ -81,13 +89,11 @@ transform mode and pick the columns using the **Columns** section.
|
||||
|
||||
The column styles allow you control how dates and numbers are formatted.
|
||||
|
||||
<img class="no-shadow" src="/img/docs/v2/Column-Options.png">
|
||||
{{< docs-imagebox img="/img/docs/v45/table_column_styles.png" class="docs-image--no-shadow" >}}
|
||||
|
||||
1. `Name or regex`: The Name or Regex field controls what columns the rule should be applied to. The regex or name filter will be matched against the column name not against column values.
|
||||
2. `Type`: The three supported types of types are `Number`, `String` and `Date`.
|
||||
3. `Title`: Title for the column, when using a Regex the title can include replacement strings like `$1`.
|
||||
4. `Format`: Specify date format. Only available when `Type` is set to `Date`.
|
||||
5. `Coloring` and `Thresholds`: Specify color mode and thresholds limits.
|
||||
6. `Unit` and `Decimals`: Specify unit and decimal precision for numbers.
|
||||
7. `Add column style rule`: Add new column rule.
|
||||
2. `Column Header`: Title for the column, when using a Regex the title can include replacement strings like `$1`.
|
||||
3. `Add column style rule`: Add new column rule.
|
||||
4. `Thresholds` and `Coloring`: Specify color mode and thresholds limits.
|
||||
5. `Type`: The three supported types of types are `Number`, `String` and `Date`. `Unit` and `Decimals`: Specify unit and decimal precision for numbers.`Format`: Specify date format for dates.
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ weight = 7
|
||||
|
||||
# Keyboard shortcuts
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v4/shortcuts.png" max-width="20rem" >}}
|
||||
{{< docs-imagebox img="/img/docs/v4/shortcuts.png" max-width="20rem" class="docs-image--right" >}}
|
||||
|
||||
Grafana v4 introduces a number of really powerful keyboard shortcuts. You can now focus a panel
|
||||
by hovering over it with your mouse. With a panel focused you can simple hit `e` to toggle panel
|
||||
|
||||
@@ -16,16 +16,13 @@ weight = -4
|
||||
|
||||
### New prometheus query editor
|
||||
|
||||
The new query editor has full syntax highlighting. As well as auto complete for metrics, functions, and range vectors.
|
||||
The new query editor has full syntax highlighting. As well as auto complete for metrics, functions, and range vectors. There is also integrated function docs right from the query editor!
|
||||
|
||||
|
||||
|
||||
There is also integrated function docs right from the query editor!
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/prometheus_query_editor_still.png" class="docs-image--block" animated-gif="/img/docs/v45/prometheus_query_editor.gif" >}}
|
||||
|
||||
### Elasticsearch: Add ad-hoc filters from the table panel
|
||||
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v45/elastic_ad_hoc_filters.png" class="docs-image--block" >}}
|
||||
|
||||
### Table cell links!
|
||||
Create column styles that turn cells into links that use the value in the cell (or other other row values) to generate a url to another dashboard or system:
|
||||
|
||||
@@ -16,7 +16,7 @@ Since Grafana automatically scales Dashboards to any resolution they're perfect
|
||||
|
||||
## Creating a Playlist
|
||||
|
||||
{{< docs-imagebox img="/img/docs/v3/playlist.png" max-width="25rem" >}}
|
||||
{{< docs-imagebox img="/img/docs/v3/playlist.png" max-width="25rem" class="docs-image--right">}}
|
||||
|
||||
The Playlist feature can be accessed from Grafana's sidemenu, in the Dashboard submenu.
|
||||
|
||||
|
||||
@@ -74,7 +74,9 @@ If you do not get an image when opening this link verify that the required font
|
||||
|
||||
### Grafana API Key
|
||||
|
||||
<img src="/img/docs/v2/orgdropdown_api_keys.png" style="width: 150px" class="right"></img>
|
||||
<!--<img src="/img/docs/v2/orgdropdown_api_keys.png" style="width: 150px" class="right"></img>-->
|
||||
{{< docs-imagebox img="/img/docs/v2/orgdropdown_api_keys.png" max-width="150px" class="docs-image--right">}}
|
||||
|
||||
You need to set the environment variable `HUBOT_GRAFANA_API_KEY` to a Grafana API Key.
|
||||
You can add these from the API Keys page which you find in the Organization dropdown.
|
||||
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@
|
||||
"company": "Grafana Labs"
|
||||
},
|
||||
"name": "grafana",
|
||||
"version": "4.5.0-beta1",
|
||||
"version": "4.5.0",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "http://github.com/grafana/grafana.git"
|
||||
|
||||
@@ -35,7 +35,7 @@ func AdminCreateUser(c *middleware.Context, form dtos.AdminCreateUserForm) {
|
||||
return
|
||||
}
|
||||
|
||||
metrics.M_Api_Admin_User_Create.Inc(1)
|
||||
metrics.M_Api_Admin_User_Create.Inc()
|
||||
|
||||
user := cmd.Result
|
||||
|
||||
|
||||
+139
-138
@@ -10,7 +10,7 @@ import (
|
||||
|
||||
// Register adds http routes
|
||||
func (hs *HttpServer) registerRoutes() {
|
||||
r := hs.macaron
|
||||
macaronR := hs.macaron
|
||||
reqSignedIn := middleware.Auth(&middleware.AuthOptions{ReqSignedIn: true})
|
||||
reqGrafanaAdmin := middleware.Auth(&middleware.AuthOptions{ReqSignedIn: true, ReqGrafanaAdmin: true})
|
||||
reqEditorRole := middleware.RoleAuth(m.ROLE_EDITOR, m.ROLE_ADMIN)
|
||||
@@ -19,7 +19,9 @@ func (hs *HttpServer) registerRoutes() {
|
||||
bind := binding.Bind
|
||||
|
||||
// automatically set HEAD for every GET
|
||||
r.SetAutoHead(true)
|
||||
macaronR.SetAutoHead(true)
|
||||
|
||||
r := newRouteRegister(middleware.RequestMetrics)
|
||||
|
||||
// not logged in views
|
||||
r.Get("/", reqSignedIn, Index)
|
||||
@@ -98,198 +100,195 @@ func (hs *HttpServer) registerRoutes() {
|
||||
r.Get("/api/login/ping", quota("session"), LoginApiPing)
|
||||
|
||||
// authed api
|
||||
r.Group("/api", func() {
|
||||
r.Group("/api", func(apiRoute RouteRegister) {
|
||||
|
||||
// user (signed in)
|
||||
r.Group("/user", func() {
|
||||
r.Get("/", wrap(GetSignedInUser))
|
||||
r.Put("/", bind(m.UpdateUserCommand{}), wrap(UpdateSignedInUser))
|
||||
r.Post("/using/:id", wrap(UserSetUsingOrg))
|
||||
r.Get("/orgs", wrap(GetSignedInUserOrgList))
|
||||
apiRoute.Group("/user", func(userRoute RouteRegister) {
|
||||
userRoute.Get("/", wrap(GetSignedInUser))
|
||||
userRoute.Put("/", bind(m.UpdateUserCommand{}), wrap(UpdateSignedInUser))
|
||||
userRoute.Post("/using/:id", wrap(UserSetUsingOrg))
|
||||
userRoute.Get("/orgs", wrap(GetSignedInUserOrgList))
|
||||
|
||||
r.Post("/stars/dashboard/:id", wrap(StarDashboard))
|
||||
r.Delete("/stars/dashboard/:id", wrap(UnstarDashboard))
|
||||
userRoute.Post("/stars/dashboard/:id", wrap(StarDashboard))
|
||||
userRoute.Delete("/stars/dashboard/:id", wrap(UnstarDashboard))
|
||||
|
||||
r.Put("/password", bind(m.ChangeUserPasswordCommand{}), wrap(ChangeUserPassword))
|
||||
r.Get("/quotas", wrap(GetUserQuotas))
|
||||
r.Put("/helpflags/:id", wrap(SetHelpFlag))
|
||||
userRoute.Put("/password", bind(m.ChangeUserPasswordCommand{}), wrap(ChangeUserPassword))
|
||||
userRoute.Get("/quotas", wrap(GetUserQuotas))
|
||||
userRoute.Put("/helpflags/:id", wrap(SetHelpFlag))
|
||||
// For dev purpose
|
||||
r.Get("/helpflags/clear", wrap(ClearHelpFlags))
|
||||
userRoute.Get("/helpflags/clear", wrap(ClearHelpFlags))
|
||||
|
||||
r.Get("/preferences", wrap(GetUserPreferences))
|
||||
r.Put("/preferences", bind(dtos.UpdatePrefsCmd{}), wrap(UpdateUserPreferences))
|
||||
userRoute.Get("/preferences", wrap(GetUserPreferences))
|
||||
userRoute.Put("/preferences", bind(dtos.UpdatePrefsCmd{}), wrap(UpdateUserPreferences))
|
||||
})
|
||||
|
||||
// users (admin permission required)
|
||||
r.Group("/users", func() {
|
||||
r.Get("/", wrap(SearchUsers))
|
||||
r.Get("/search", wrap(SearchUsersWithPaging))
|
||||
r.Get("/:id", wrap(GetUserById))
|
||||
r.Get("/:id/orgs", wrap(GetUserOrgList))
|
||||
apiRoute.Group("/users", func(usersRoute RouteRegister) {
|
||||
usersRoute.Get("/", wrap(SearchUsers))
|
||||
usersRoute.Get("/search", wrap(SearchUsersWithPaging))
|
||||
usersRoute.Get("/:id", wrap(GetUserById))
|
||||
usersRoute.Get("/:id/orgs", wrap(GetUserOrgList))
|
||||
// query parameters /users/lookup?loginOrEmail=admin@example.com
|
||||
r.Get("/lookup", wrap(GetUserByLoginOrEmail))
|
||||
r.Put("/:id", bind(m.UpdateUserCommand{}), wrap(UpdateUser))
|
||||
r.Post("/:id/using/:orgId", wrap(UpdateUserActiveOrg))
|
||||
usersRoute.Get("/lookup", wrap(GetUserByLoginOrEmail))
|
||||
usersRoute.Put("/:id", bind(m.UpdateUserCommand{}), wrap(UpdateUser))
|
||||
usersRoute.Post("/:id/using/:orgId", wrap(UpdateUserActiveOrg))
|
||||
}, reqGrafanaAdmin)
|
||||
|
||||
// org information available to all users.
|
||||
r.Group("/org", func() {
|
||||
r.Get("/", wrap(GetOrgCurrent))
|
||||
r.Get("/quotas", wrap(GetOrgQuotas))
|
||||
apiRoute.Group("/org", func(orgRoute RouteRegister) {
|
||||
orgRoute.Get("/", wrap(GetOrgCurrent))
|
||||
orgRoute.Get("/quotas", wrap(GetOrgQuotas))
|
||||
})
|
||||
|
||||
// current org
|
||||
r.Group("/org", func() {
|
||||
r.Put("/", bind(dtos.UpdateOrgForm{}), wrap(UpdateOrgCurrent))
|
||||
r.Put("/address", bind(dtos.UpdateOrgAddressForm{}), wrap(UpdateOrgAddressCurrent))
|
||||
r.Post("/users", quota("user"), bind(m.AddOrgUserCommand{}), wrap(AddOrgUserToCurrentOrg))
|
||||
r.Get("/users", wrap(GetOrgUsersForCurrentOrg))
|
||||
r.Patch("/users/:userId", bind(m.UpdateOrgUserCommand{}), wrap(UpdateOrgUserForCurrentOrg))
|
||||
r.Delete("/users/:userId", wrap(RemoveOrgUserForCurrentOrg))
|
||||
apiRoute.Group("/org", func(orgRoute RouteRegister) {
|
||||
orgRoute.Put("/", bind(dtos.UpdateOrgForm{}), wrap(UpdateOrgCurrent))
|
||||
orgRoute.Put("/address", bind(dtos.UpdateOrgAddressForm{}), wrap(UpdateOrgAddressCurrent))
|
||||
orgRoute.Post("/users", quota("user"), bind(m.AddOrgUserCommand{}), wrap(AddOrgUserToCurrentOrg))
|
||||
orgRoute.Get("/users", wrap(GetOrgUsersForCurrentOrg))
|
||||
orgRoute.Patch("/users/:userId", bind(m.UpdateOrgUserCommand{}), wrap(UpdateOrgUserForCurrentOrg))
|
||||
orgRoute.Delete("/users/:userId", wrap(RemoveOrgUserForCurrentOrg))
|
||||
|
||||
// invites
|
||||
r.Get("/invites", wrap(GetPendingOrgInvites))
|
||||
r.Post("/invites", quota("user"), bind(dtos.AddInviteForm{}), wrap(AddOrgInvite))
|
||||
r.Patch("/invites/:code/revoke", wrap(RevokeInvite))
|
||||
orgRoute.Get("/invites", wrap(GetPendingOrgInvites))
|
||||
orgRoute.Post("/invites", quota("user"), bind(dtos.AddInviteForm{}), wrap(AddOrgInvite))
|
||||
orgRoute.Patch("/invites/:code/revoke", wrap(RevokeInvite))
|
||||
|
||||
// prefs
|
||||
r.Get("/preferences", wrap(GetOrgPreferences))
|
||||
r.Put("/preferences", bind(dtos.UpdatePrefsCmd{}), wrap(UpdateOrgPreferences))
|
||||
orgRoute.Get("/preferences", wrap(GetOrgPreferences))
|
||||
orgRoute.Put("/preferences", bind(dtos.UpdatePrefsCmd{}), wrap(UpdateOrgPreferences))
|
||||
}, reqOrgAdmin)
|
||||
|
||||
// create new org
|
||||
r.Post("/orgs", quota("org"), bind(m.CreateOrgCommand{}), wrap(CreateOrg))
|
||||
apiRoute.Post("/orgs", quota("org"), bind(m.CreateOrgCommand{}), wrap(CreateOrg))
|
||||
|
||||
// search all orgs
|
||||
r.Get("/orgs", reqGrafanaAdmin, wrap(SearchOrgs))
|
||||
apiRoute.Get("/orgs", reqGrafanaAdmin, wrap(SearchOrgs))
|
||||
|
||||
// orgs (admin routes)
|
||||
r.Group("/orgs/:orgId", func() {
|
||||
r.Get("/", wrap(GetOrgById))
|
||||
r.Put("/", bind(dtos.UpdateOrgForm{}), wrap(UpdateOrg))
|
||||
r.Put("/address", bind(dtos.UpdateOrgAddressForm{}), wrap(UpdateOrgAddress))
|
||||
r.Delete("/", wrap(DeleteOrgById))
|
||||
r.Get("/users", wrap(GetOrgUsers))
|
||||
r.Post("/users", bind(m.AddOrgUserCommand{}), wrap(AddOrgUser))
|
||||
r.Patch("/users/:userId", bind(m.UpdateOrgUserCommand{}), wrap(UpdateOrgUser))
|
||||
r.Delete("/users/:userId", wrap(RemoveOrgUser))
|
||||
r.Get("/quotas", wrap(GetOrgQuotas))
|
||||
r.Put("/quotas/:target", bind(m.UpdateOrgQuotaCmd{}), wrap(UpdateOrgQuota))
|
||||
apiRoute.Group("/orgs/:orgId", func(orgsRoute RouteRegister) {
|
||||
orgsRoute.Get("/", wrap(GetOrgById))
|
||||
orgsRoute.Put("/", bind(dtos.UpdateOrgForm{}), wrap(UpdateOrg))
|
||||
orgsRoute.Put("/address", bind(dtos.UpdateOrgAddressForm{}), wrap(UpdateOrgAddress))
|
||||
orgsRoute.Delete("/", wrap(DeleteOrgById))
|
||||
orgsRoute.Get("/users", wrap(GetOrgUsers))
|
||||
orgsRoute.Post("/users", bind(m.AddOrgUserCommand{}), wrap(AddOrgUser))
|
||||
orgsRoute.Patch("/users/:userId", bind(m.UpdateOrgUserCommand{}), wrap(UpdateOrgUser))
|
||||
orgsRoute.Delete("/users/:userId", wrap(RemoveOrgUser))
|
||||
orgsRoute.Get("/quotas", wrap(GetOrgQuotas))
|
||||
orgsRoute.Put("/quotas/:target", bind(m.UpdateOrgQuotaCmd{}), wrap(UpdateOrgQuota))
|
||||
}, reqGrafanaAdmin)
|
||||
|
||||
// orgs (admin routes)
|
||||
r.Group("/orgs/name/:name", func() {
|
||||
r.Get("/", wrap(GetOrgByName))
|
||||
apiRoute.Group("/orgs/name/:name", func(orgsRoute RouteRegister) {
|
||||
orgsRoute.Get("/", wrap(GetOrgByName))
|
||||
}, reqGrafanaAdmin)
|
||||
|
||||
// auth api keys
|
||||
r.Group("/auth/keys", func() {
|
||||
r.Get("/", wrap(GetApiKeys))
|
||||
r.Post("/", quota("api_key"), bind(m.AddApiKeyCommand{}), wrap(AddApiKey))
|
||||
r.Delete("/:id", wrap(DeleteApiKey))
|
||||
apiRoute.Group("/auth/keys", func(keysRoute RouteRegister) {
|
||||
keysRoute.Get("/", wrap(GetApiKeys))
|
||||
keysRoute.Post("/", quota("api_key"), bind(m.AddApiKeyCommand{}), wrap(AddApiKey))
|
||||
keysRoute.Delete("/:id", wrap(DeleteApiKey))
|
||||
}, reqOrgAdmin)
|
||||
|
||||
// Preferences
|
||||
r.Group("/preferences", func() {
|
||||
r.Post("/set-home-dash", bind(m.SavePreferencesCommand{}), wrap(SetHomeDashboard))
|
||||
apiRoute.Group("/preferences", func(prefRoute RouteRegister) {
|
||||
prefRoute.Post("/set-home-dash", bind(m.SavePreferencesCommand{}), wrap(SetHomeDashboard))
|
||||
})
|
||||
|
||||
// Data sources
|
||||
r.Group("/datasources", func() {
|
||||
r.Get("/", wrap(GetDataSources))
|
||||
r.Post("/", quota("data_source"), bind(m.AddDataSourceCommand{}), AddDataSource)
|
||||
r.Put("/:id", bind(m.UpdateDataSourceCommand{}), wrap(UpdateDataSource))
|
||||
r.Delete("/:id", DeleteDataSourceById)
|
||||
r.Delete("/name/:name", DeleteDataSourceByName)
|
||||
r.Get("/:id", wrap(GetDataSourceById))
|
||||
r.Get("/name/:name", wrap(GetDataSourceByName))
|
||||
apiRoute.Group("/datasources", func(datasourceRoute RouteRegister) {
|
||||
datasourceRoute.Get("/", wrap(GetDataSources))
|
||||
datasourceRoute.Post("/", quota("data_source"), bind(m.AddDataSourceCommand{}), AddDataSource)
|
||||
datasourceRoute.Put("/:id", bind(m.UpdateDataSourceCommand{}), wrap(UpdateDataSource))
|
||||
datasourceRoute.Delete("/:id", DeleteDataSourceById)
|
||||
datasourceRoute.Delete("/name/:name", DeleteDataSourceByName)
|
||||
datasourceRoute.Get("/:id", wrap(GetDataSourceById))
|
||||
datasourceRoute.Get("/name/:name", wrap(GetDataSourceByName))
|
||||
}, reqOrgAdmin)
|
||||
|
||||
r.Get("/datasources/id/:name", wrap(GetDataSourceIdByName), reqSignedIn)
|
||||
apiRoute.Get("/datasources/id/:name", wrap(GetDataSourceIdByName), reqSignedIn)
|
||||
|
||||
r.Get("/plugins", wrap(GetPluginList))
|
||||
r.Get("/plugins/:pluginId/settings", wrap(GetPluginSettingById))
|
||||
r.Get("/plugins/:pluginId/markdown/:name", wrap(GetPluginMarkdown))
|
||||
apiRoute.Get("/plugins", wrap(GetPluginList))
|
||||
apiRoute.Get("/plugins/:pluginId/settings", wrap(GetPluginSettingById))
|
||||
apiRoute.Get("/plugins/:pluginId/markdown/:name", wrap(GetPluginMarkdown))
|
||||
|
||||
r.Group("/plugins", func() {
|
||||
r.Get("/:pluginId/dashboards/", wrap(GetPluginDashboards))
|
||||
r.Post("/:pluginId/settings", bind(m.UpdatePluginSettingCmd{}), wrap(UpdatePluginSetting))
|
||||
apiRoute.Group("/plugins", func(pluginRoute RouteRegister) {
|
||||
pluginRoute.Get("/:pluginId/dashboards/", wrap(GetPluginDashboards))
|
||||
pluginRoute.Post("/:pluginId/settings", bind(m.UpdatePluginSettingCmd{}), wrap(UpdatePluginSetting))
|
||||
}, reqOrgAdmin)
|
||||
|
||||
r.Get("/frontend/settings/", GetFrontendSettings)
|
||||
r.Any("/datasources/proxy/:id/*", reqSignedIn, hs.ProxyDataSourceRequest)
|
||||
r.Any("/datasources/proxy/:id", reqSignedIn, hs.ProxyDataSourceRequest)
|
||||
apiRoute.Get("/frontend/settings/", GetFrontendSettings)
|
||||
apiRoute.Any("/datasources/proxy/:id/*", reqSignedIn, hs.ProxyDataSourceRequest)
|
||||
apiRoute.Any("/datasources/proxy/:id", reqSignedIn, hs.ProxyDataSourceRequest)
|
||||
|
||||
// Dashboard
|
||||
r.Group("/dashboards", func() {
|
||||
r.Get("/db/:slug", GetDashboard)
|
||||
r.Delete("/db/:slug", reqEditorRole, DeleteDashboard)
|
||||
apiRoute.Group("/dashboards", func(dashboardRoute RouteRegister) {
|
||||
dashboardRoute.Get("/db/:slug", GetDashboard)
|
||||
dashboardRoute.Delete("/db/:slug", reqEditorRole, DeleteDashboard)
|
||||
|
||||
r.Get("/id/:dashboardId/versions", wrap(GetDashboardVersions))
|
||||
r.Get("/id/:dashboardId/versions/:id", wrap(GetDashboardVersion))
|
||||
r.Post("/id/:dashboardId/restore", reqEditorRole, bind(dtos.RestoreDashboardVersionCommand{}), wrap(RestoreDashboardVersion))
|
||||
dashboardRoute.Get("/id/:dashboardId/versions", wrap(GetDashboardVersions))
|
||||
dashboardRoute.Get("/id/:dashboardId/versions/:id", wrap(GetDashboardVersion))
|
||||
dashboardRoute.Post("/id/:dashboardId/restore", reqEditorRole, bind(dtos.RestoreDashboardVersionCommand{}), wrap(RestoreDashboardVersion))
|
||||
|
||||
r.Post("/calculate-diff", bind(dtos.CalculateDiffOptions{}), wrap(CalculateDashboardDiff))
|
||||
dashboardRoute.Post("/calculate-diff", bind(dtos.CalculateDiffOptions{}), wrap(CalculateDashboardDiff))
|
||||
|
||||
r.Post("/db", reqEditorRole, bind(m.SaveDashboardCommand{}), wrap(PostDashboard))
|
||||
r.Get("/file/:file", GetDashboardFromJsonFile)
|
||||
r.Get("/home", wrap(GetHomeDashboard))
|
||||
r.Get("/tags", GetDashboardTags)
|
||||
r.Post("/import", bind(dtos.ImportDashboardCommand{}), wrap(ImportDashboard))
|
||||
dashboardRoute.Post("/db", reqEditorRole, bind(m.SaveDashboardCommand{}), wrap(PostDashboard))
|
||||
dashboardRoute.Get("/file/:file", GetDashboardFromJsonFile)
|
||||
dashboardRoute.Get("/home", wrap(GetHomeDashboard))
|
||||
dashboardRoute.Get("/tags", GetDashboardTags)
|
||||
dashboardRoute.Post("/import", bind(dtos.ImportDashboardCommand{}), wrap(ImportDashboard))
|
||||
})
|
||||
|
||||
// Dashboard snapshots
|
||||
r.Group("/dashboard/snapshots", func() {
|
||||
r.Get("/", wrap(SearchDashboardSnapshots))
|
||||
apiRoute.Group("/dashboard/snapshots", func(dashboardRoute RouteRegister) {
|
||||
dashboardRoute.Get("/", wrap(SearchDashboardSnapshots))
|
||||
})
|
||||
|
||||
// Playlist
|
||||
r.Group("/playlists", func() {
|
||||
r.Get("/", wrap(SearchPlaylists))
|
||||
r.Get("/:id", ValidateOrgPlaylist, wrap(GetPlaylist))
|
||||
r.Get("/:id/items", ValidateOrgPlaylist, wrap(GetPlaylistItems))
|
||||
r.Get("/:id/dashboards", ValidateOrgPlaylist, wrap(GetPlaylistDashboards))
|
||||
r.Delete("/:id", reqEditorRole, ValidateOrgPlaylist, wrap(DeletePlaylist))
|
||||
r.Put("/:id", reqEditorRole, bind(m.UpdatePlaylistCommand{}), ValidateOrgPlaylist, wrap(UpdatePlaylist))
|
||||
r.Post("/", reqEditorRole, bind(m.CreatePlaylistCommand{}), wrap(CreatePlaylist))
|
||||
apiRoute.Group("/playlists", func(playlistRoute RouteRegister) {
|
||||
playlistRoute.Get("/", wrap(SearchPlaylists))
|
||||
playlistRoute.Get("/:id", ValidateOrgPlaylist, wrap(GetPlaylist))
|
||||
playlistRoute.Get("/:id/items", ValidateOrgPlaylist, wrap(GetPlaylistItems))
|
||||
playlistRoute.Get("/:id/dashboards", ValidateOrgPlaylist, wrap(GetPlaylistDashboards))
|
||||
playlistRoute.Delete("/:id", reqEditorRole, ValidateOrgPlaylist, wrap(DeletePlaylist))
|
||||
playlistRoute.Put("/:id", reqEditorRole, bind(m.UpdatePlaylistCommand{}), ValidateOrgPlaylist, wrap(UpdatePlaylist))
|
||||
playlistRoute.Post("/", reqEditorRole, bind(m.CreatePlaylistCommand{}), wrap(CreatePlaylist))
|
||||
})
|
||||
|
||||
// Search
|
||||
r.Get("/search/", Search)
|
||||
apiRoute.Get("/search/", Search)
|
||||
|
||||
// metrics
|
||||
r.Post("/tsdb/query", bind(dtos.MetricRequest{}), wrap(QueryMetrics))
|
||||
r.Get("/tsdb/testdata/scenarios", wrap(GetTestDataScenarios))
|
||||
r.Get("/tsdb/testdata/gensql", reqGrafanaAdmin, wrap(GenerateSqlTestData))
|
||||
r.Get("/tsdb/testdata/random-walk", wrap(GetTestDataRandomWalk))
|
||||
apiRoute.Post("/tsdb/query", bind(dtos.MetricRequest{}), wrap(QueryMetrics))
|
||||
apiRoute.Get("/tsdb/testdata/scenarios", wrap(GetTestDataScenarios))
|
||||
apiRoute.Get("/tsdb/testdata/gensql", reqGrafanaAdmin, wrap(GenerateSqlTestData))
|
||||
apiRoute.Get("/tsdb/testdata/random-walk", wrap(GetTestDataRandomWalk))
|
||||
|
||||
// metrics
|
||||
r.Get("/metrics", wrap(GetInternalMetrics))
|
||||
|
||||
r.Group("/alerts", func() {
|
||||
r.Post("/test", bind(dtos.AlertTestCommand{}), wrap(AlertTest))
|
||||
r.Post("/:alertId/pause", bind(dtos.PauseAlertCommand{}), wrap(PauseAlert), reqEditorRole)
|
||||
r.Get("/:alertId", ValidateOrgAlert, wrap(GetAlert))
|
||||
r.Get("/", wrap(GetAlerts))
|
||||
r.Get("/states-for-dashboard", wrap(GetAlertStatesForDashboard))
|
||||
apiRoute.Group("/alerts", func(alertsRoute RouteRegister) {
|
||||
alertsRoute.Post("/test", bind(dtos.AlertTestCommand{}), wrap(AlertTest))
|
||||
alertsRoute.Post("/:alertId/pause", bind(dtos.PauseAlertCommand{}), wrap(PauseAlert), reqEditorRole)
|
||||
alertsRoute.Get("/:alertId", ValidateOrgAlert, wrap(GetAlert))
|
||||
alertsRoute.Get("/", wrap(GetAlerts))
|
||||
alertsRoute.Get("/states-for-dashboard", wrap(GetAlertStatesForDashboard))
|
||||
})
|
||||
|
||||
r.Get("/alert-notifications", wrap(GetAlertNotifications))
|
||||
r.Get("/alert-notifiers", wrap(GetAlertNotifiers))
|
||||
apiRoute.Get("/alert-notifications", wrap(GetAlertNotifications))
|
||||
apiRoute.Get("/alert-notifiers", wrap(GetAlertNotifiers))
|
||||
|
||||
r.Group("/alert-notifications", func() {
|
||||
r.Post("/test", bind(dtos.NotificationTestCommand{}), wrap(NotificationTest))
|
||||
r.Post("/", bind(m.CreateAlertNotificationCommand{}), wrap(CreateAlertNotification))
|
||||
r.Put("/:notificationId", bind(m.UpdateAlertNotificationCommand{}), wrap(UpdateAlertNotification))
|
||||
r.Get("/:notificationId", wrap(GetAlertNotificationById))
|
||||
r.Delete("/:notificationId", wrap(DeleteAlertNotification))
|
||||
apiRoute.Group("/alert-notifications", func(alertNotifications RouteRegister) {
|
||||
alertNotifications.Post("/test", bind(dtos.NotificationTestCommand{}), wrap(NotificationTest))
|
||||
alertNotifications.Post("/", bind(m.CreateAlertNotificationCommand{}), wrap(CreateAlertNotification))
|
||||
alertNotifications.Put("/:notificationId", bind(m.UpdateAlertNotificationCommand{}), wrap(UpdateAlertNotification))
|
||||
alertNotifications.Get("/:notificationId", wrap(GetAlertNotificationById))
|
||||
alertNotifications.Delete("/:notificationId", wrap(DeleteAlertNotification))
|
||||
}, reqEditorRole)
|
||||
|
||||
r.Get("/annotations", wrap(GetAnnotations))
|
||||
r.Post("/annotations/mass-delete", reqOrgAdmin, bind(dtos.DeleteAnnotationsCmd{}), wrap(DeleteAnnotations))
|
||||
apiRoute.Get("/annotations", wrap(GetAnnotations))
|
||||
apiRoute.Post("/annotations/mass-delete", reqOrgAdmin, bind(dtos.DeleteAnnotationsCmd{}), wrap(DeleteAnnotations))
|
||||
|
||||
r.Group("/annotations", func() {
|
||||
r.Post("/", bind(dtos.PostAnnotationsCmd{}), wrap(PostAnnotation))
|
||||
apiRoute.Group("/annotations", func(annotationsRoute RouteRegister) {
|
||||
annotationsRoute.Post("/", bind(dtos.PostAnnotationsCmd{}), wrap(PostAnnotation))
|
||||
}, reqEditorRole)
|
||||
|
||||
// error test
|
||||
@@ -298,16 +297,16 @@ func (hs *HttpServer) registerRoutes() {
|
||||
}, reqSignedIn)
|
||||
|
||||
// admin api
|
||||
r.Group("/api/admin", func() {
|
||||
r.Get("/settings", AdminGetSettings)
|
||||
r.Post("/users", bind(dtos.AdminCreateUserForm{}), AdminCreateUser)
|
||||
r.Put("/users/:id/password", bind(dtos.AdminUpdateUserPasswordForm{}), AdminUpdateUserPassword)
|
||||
r.Put("/users/:id/permissions", bind(dtos.AdminUpdateUserPermissionsForm{}), AdminUpdateUserPermissions)
|
||||
r.Delete("/users/:id", AdminDeleteUser)
|
||||
r.Get("/users/:id/quotas", wrap(GetUserQuotas))
|
||||
r.Put("/users/:id/quotas/:target", bind(m.UpdateUserQuotaCmd{}), wrap(UpdateUserQuota))
|
||||
r.Get("/stats", AdminGetStats)
|
||||
r.Post("/pause-all-alerts", bind(dtos.PauseAllAlertsCommand{}), wrap(PauseAllAlerts))
|
||||
r.Group("/api/admin", func(adminRoute RouteRegister) {
|
||||
adminRoute.Get("/settings", AdminGetSettings)
|
||||
adminRoute.Post("/users", bind(dtos.AdminCreateUserForm{}), AdminCreateUser)
|
||||
adminRoute.Put("/users/:id/password", bind(dtos.AdminUpdateUserPasswordForm{}), AdminUpdateUserPassword)
|
||||
adminRoute.Put("/users/:id/permissions", bind(dtos.AdminUpdateUserPermissionsForm{}), AdminUpdateUserPermissions)
|
||||
adminRoute.Delete("/users/:id", AdminDeleteUser)
|
||||
adminRoute.Get("/users/:id/quotas", wrap(GetUserQuotas))
|
||||
adminRoute.Put("/users/:id/quotas/:target", bind(m.UpdateUserQuotaCmd{}), wrap(UpdateUserQuota))
|
||||
adminRoute.Get("/stats", AdminGetStats)
|
||||
adminRoute.Post("/pause-all-alerts", bind(dtos.PauseAllAlertsCommand{}), wrap(PauseAllAlerts))
|
||||
}, reqGrafanaAdmin)
|
||||
|
||||
// rendering
|
||||
@@ -326,7 +325,9 @@ func (hs *HttpServer) registerRoutes() {
|
||||
// streams
|
||||
//r.Post("/api/streams/push", reqSignedIn, bind(dtos.StreamMessage{}), liveConn.PushToStream)
|
||||
|
||||
InitAppPluginRoutes(r)
|
||||
r.Register(macaronR)
|
||||
|
||||
r.NotFound(NotFoundHandler)
|
||||
InitAppPluginRoutes(macaronR)
|
||||
|
||||
macaronR.NotFound(NotFoundHandler)
|
||||
}
|
||||
|
||||
@@ -266,7 +266,7 @@ func handleGetMetricStatistics(req *cwRequest, c *middleware.Context) {
|
||||
c.JsonApiErr(500, "Unable to call AWS API", err)
|
||||
return
|
||||
}
|
||||
metrics.M_Aws_CloudWatch_GetMetricStatistics.Inc(1)
|
||||
metrics.M_Aws_CloudWatch_GetMetricStatistics.Inc()
|
||||
|
||||
c.JSON(200, resp)
|
||||
}
|
||||
@@ -302,7 +302,7 @@ func handleListMetrics(req *cwRequest, c *middleware.Context) {
|
||||
var resp cloudwatch.ListMetricsOutput
|
||||
err = svc.ListMetricsPages(params,
|
||||
func(page *cloudwatch.ListMetricsOutput, lastPage bool) bool {
|
||||
metrics.M_Aws_CloudWatch_ListMetrics.Inc(1)
|
||||
metrics.M_Aws_CloudWatch_ListMetrics.Inc()
|
||||
metrics, _ := awsutil.ValuesAtPath(page, "Metrics")
|
||||
for _, metric := range metrics {
|
||||
resp.Metrics = append(resp.Metrics, metric.(*cloudwatch.Metric))
|
||||
|
||||
@@ -275,7 +275,7 @@ func getAllMetrics(cwData *datasourceInfo) (cloudwatch.ListMetricsOutput, error)
|
||||
var resp cloudwatch.ListMetricsOutput
|
||||
err = svc.ListMetricsPages(params,
|
||||
func(page *cloudwatch.ListMetricsOutput, lastPage bool) bool {
|
||||
metrics.M_Aws_CloudWatch_ListMetrics.Inc(1)
|
||||
metrics.M_Aws_CloudWatch_ListMetrics.Inc()
|
||||
metrics, _ := awsutil.ValuesAtPath(page, "Metrics")
|
||||
for _, metric := range metrics {
|
||||
resp.Metrics = append(resp.Metrics, metric.(*cloudwatch.Metric))
|
||||
|
||||
@@ -34,13 +34,13 @@ func CreateDashboardSnapshot(c *middleware.Context, cmd m.CreateDashboardSnapsho
|
||||
|
||||
cmd.OrgId = -1
|
||||
cmd.UserId = -1
|
||||
metrics.M_Api_Dashboard_Snapshot_External.Inc(1)
|
||||
metrics.M_Api_Dashboard_Snapshot_External.Inc()
|
||||
} else {
|
||||
cmd.Key = util.GetRandomString(32)
|
||||
cmd.DeleteKey = util.GetRandomString(32)
|
||||
cmd.OrgId = c.OrgId
|
||||
cmd.UserId = c.UserId
|
||||
metrics.M_Api_Dashboard_Snapshot_Create.Inc(1)
|
||||
metrics.M_Api_Dashboard_Snapshot_Create.Inc()
|
||||
}
|
||||
|
||||
if err := bus.Dispatch(&cmd); err != nil {
|
||||
@@ -84,7 +84,7 @@ func GetDashboardSnapshot(c *middleware.Context) {
|
||||
},
|
||||
}
|
||||
|
||||
metrics.M_Api_Dashboard_Snapshot_Get.Inc(1)
|
||||
metrics.M_Api_Dashboard_Snapshot_Get.Inc()
|
||||
|
||||
c.Resp.Header().Set("Cache-Control", "public, max-age=3600")
|
||||
c.JSON(200, dto)
|
||||
|
||||
+11
-1
@@ -11,6 +11,8 @@ import (
|
||||
"path"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
|
||||
gocache "github.com/patrickmn/go-cache"
|
||||
macaron "gopkg.in/macaron.v1"
|
||||
|
||||
@@ -165,9 +167,9 @@ func (hs *HttpServer) newMacaron() *macaron.Macaron {
|
||||
}))
|
||||
|
||||
m.Use(hs.healthHandler)
|
||||
m.Use(hs.metricsEndpoint)
|
||||
m.Use(middleware.GetContextHandler())
|
||||
m.Use(middleware.Sessioner(&setting.SessionOptions))
|
||||
m.Use(middleware.RequestMetrics())
|
||||
m.Use(middleware.OrgRedirect())
|
||||
|
||||
// needs to be after context handler
|
||||
@@ -180,6 +182,14 @@ func (hs *HttpServer) newMacaron() *macaron.Macaron {
|
||||
return m
|
||||
}
|
||||
|
||||
func (hs *HttpServer) metricsEndpoint(ctx *macaron.Context) {
|
||||
if ctx.Req.Method != "GET" || ctx.Req.URL.Path != "/metrics" {
|
||||
return
|
||||
}
|
||||
|
||||
promhttp.Handler().ServeHTTP(ctx.Resp, ctx.Req.Request)
|
||||
}
|
||||
|
||||
func (hs *HttpServer) healthHandler(ctx *macaron.Context) {
|
||||
if ctx.Req.Method != "GET" || ctx.Req.URL.Path != "/api/health" {
|
||||
return
|
||||
|
||||
+1
-1
@@ -127,7 +127,7 @@ func LoginPost(c *middleware.Context, cmd dtos.LoginCommand) Response {
|
||||
c.SetCookie("redirect_to", "", -1, setting.AppSubUrl+"/")
|
||||
}
|
||||
|
||||
metrics.M_Api_Login_Post.Inc(1)
|
||||
metrics.M_Api_Login_Post.Inc()
|
||||
|
||||
return Json(200, result)
|
||||
}
|
||||
|
||||
@@ -186,7 +186,7 @@ func OAuthLogin(ctx *middleware.Context) {
|
||||
// login
|
||||
loginUserWithUser(userQuery.Result, ctx)
|
||||
|
||||
metrics.M_Api_Login_OAuth.Inc(1)
|
||||
metrics.M_Api_Login_OAuth.Inc()
|
||||
|
||||
if redirectTo, _ := url.QueryUnescape(ctx.GetCookie("redirect_to")); len(redirectTo) > 0 {
|
||||
ctx.SetCookie("redirect_to", "", -1, setting.AppSubUrl+"/")
|
||||
|
||||
@@ -2,13 +2,10 @@ package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/grafana/grafana/pkg/api/dtos"
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
"github.com/grafana/grafana/pkg/middleware"
|
||||
"github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/tsdb"
|
||||
@@ -79,58 +76,6 @@ func GetTestDataScenarios(c *middleware.Context) Response {
|
||||
return Json(200, &result)
|
||||
}
|
||||
|
||||
func GetInternalMetrics(c *middleware.Context) Response {
|
||||
if metrics.UseNilMetrics {
|
||||
return Json(200, util.DynMap{"message": "Metrics disabled"})
|
||||
}
|
||||
|
||||
snapshots := metrics.MetricStats.GetSnapshots()
|
||||
|
||||
resp := make(map[string]interface{})
|
||||
|
||||
for _, m := range snapshots {
|
||||
metricName := m.Name() + m.StringifyTags()
|
||||
|
||||
switch metric := m.(type) {
|
||||
case metrics.Gauge:
|
||||
resp[metricName] = map[string]interface{}{
|
||||
"value": metric.Value(),
|
||||
}
|
||||
case metrics.Counter:
|
||||
resp[metricName] = map[string]interface{}{
|
||||
"count": metric.Count(),
|
||||
}
|
||||
case metrics.Timer:
|
||||
percentiles := metric.Percentiles([]float64{0.25, 0.75, 0.90, 0.99})
|
||||
resp[metricName] = map[string]interface{}{
|
||||
"count": metric.Count(),
|
||||
"min": metric.Min(),
|
||||
"max": metric.Max(),
|
||||
"mean": metric.Mean(),
|
||||
"std": metric.StdDev(),
|
||||
"p25": percentiles[0],
|
||||
"p75": percentiles[1],
|
||||
"p90": percentiles[2],
|
||||
"p99": percentiles[3],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var b []byte
|
||||
var err error
|
||||
if b, err = json.MarshalIndent(resp, "", " "); err != nil {
|
||||
return ApiError(500, "body json marshal", err)
|
||||
}
|
||||
|
||||
return &NormalResponse{
|
||||
body: b,
|
||||
status: 200,
|
||||
header: http.Header{
|
||||
"Content-Type": []string{"application/json"},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Genereates a index out of range error
|
||||
func GenerateError(c *middleware.Context) Response {
|
||||
var array []string
|
||||
|
||||
+1
-1
@@ -89,7 +89,7 @@ func CreateOrg(c *middleware.Context, cmd m.CreateOrgCommand) Response {
|
||||
return ApiError(500, "Failed to create organization", err)
|
||||
}
|
||||
|
||||
metrics.M_Api_Org_Create.Inc(1)
|
||||
metrics.M_Api_Org_Create.Inc()
|
||||
|
||||
return Json(200, &util.DynMap{
|
||||
"orgId": cmd.Result.Id,
|
||||
|
||||
@@ -187,8 +187,8 @@ func CompleteInvite(c *middleware.Context, completeInvite dtos.CompleteInviteFor
|
||||
|
||||
loginUserWithUser(user, c)
|
||||
|
||||
metrics.M_Api_User_SignUpCompleted.Inc(1)
|
||||
metrics.M_Api_User_SignUpInvite.Inc(1)
|
||||
metrics.M_Api_User_SignUpCompleted.Inc()
|
||||
metrics.M_Api_User_SignUpInvite.Inc()
|
||||
|
||||
return ApiSuccess("User created and logged in")
|
||||
}
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
macaron "gopkg.in/macaron.v1"
|
||||
)
|
||||
|
||||
type Router interface {
|
||||
Handle(method, pattern string, handlers []macaron.Handler) *macaron.Route
|
||||
}
|
||||
|
||||
type RouteRegister interface {
|
||||
Get(string, ...macaron.Handler)
|
||||
Post(string, ...macaron.Handler)
|
||||
Delete(string, ...macaron.Handler)
|
||||
Put(string, ...macaron.Handler)
|
||||
Patch(string, ...macaron.Handler)
|
||||
Any(string, ...macaron.Handler)
|
||||
|
||||
Group(string, func(RouteRegister), ...macaron.Handler)
|
||||
|
||||
Register(Router) *macaron.Router
|
||||
}
|
||||
|
||||
type RegisterNamedMiddleware func(name string) macaron.Handler
|
||||
|
||||
func newRouteRegister(namedMiddleware ...RegisterNamedMiddleware) RouteRegister {
|
||||
return &routeRegister{
|
||||
prefix: "",
|
||||
routes: []route{},
|
||||
subfixHandlers: []macaron.Handler{},
|
||||
namedMiddleware: namedMiddleware,
|
||||
}
|
||||
}
|
||||
|
||||
type route struct {
|
||||
method string
|
||||
pattern string
|
||||
handlers []macaron.Handler
|
||||
}
|
||||
|
||||
type routeRegister struct {
|
||||
prefix string
|
||||
subfixHandlers []macaron.Handler
|
||||
namedMiddleware []RegisterNamedMiddleware
|
||||
routes []route
|
||||
groups []*routeRegister
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Group(pattern string, fn func(rr RouteRegister), handlers ...macaron.Handler) {
|
||||
group := &routeRegister{
|
||||
prefix: rr.prefix + pattern,
|
||||
subfixHandlers: append(rr.subfixHandlers, handlers...),
|
||||
routes: []route{},
|
||||
namedMiddleware: rr.namedMiddleware,
|
||||
}
|
||||
|
||||
fn(group)
|
||||
rr.groups = append(rr.groups, group)
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Register(router Router) *macaron.Router {
|
||||
for _, r := range rr.routes {
|
||||
router.Handle(r.method, r.pattern, r.handlers)
|
||||
}
|
||||
|
||||
for _, g := range rr.groups {
|
||||
g.Register(router)
|
||||
}
|
||||
|
||||
return &macaron.Router{}
|
||||
}
|
||||
|
||||
func (rr *routeRegister) route(pattern, method string, handlers ...macaron.Handler) {
|
||||
//inject tracing
|
||||
|
||||
h := make([]macaron.Handler, 0)
|
||||
for _, fn := range rr.namedMiddleware {
|
||||
h = append(h, fn(pattern))
|
||||
}
|
||||
|
||||
h = append(h, rr.subfixHandlers...)
|
||||
h = append(h, handlers...)
|
||||
|
||||
rr.routes = append(rr.routes, route{
|
||||
method: method,
|
||||
pattern: rr.prefix + pattern,
|
||||
handlers: h,
|
||||
})
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Get(pattern string, handlers ...macaron.Handler) {
|
||||
rr.route(pattern, http.MethodGet, handlers...)
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Post(pattern string, handlers ...macaron.Handler) {
|
||||
rr.route(pattern, http.MethodPost, handlers...)
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Delete(pattern string, handlers ...macaron.Handler) {
|
||||
rr.route(pattern, http.MethodDelete, handlers...)
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Put(pattern string, handlers ...macaron.Handler) {
|
||||
rr.route(pattern, http.MethodPut, handlers...)
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Patch(pattern string, handlers ...macaron.Handler) {
|
||||
rr.route(pattern, http.MethodPatch, handlers...)
|
||||
}
|
||||
|
||||
func (rr *routeRegister) Any(pattern string, handlers ...macaron.Handler) {
|
||||
rr.route(pattern, "*", handlers...)
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"testing"
|
||||
|
||||
macaron "gopkg.in/macaron.v1"
|
||||
)
|
||||
|
||||
type fakeRouter struct {
|
||||
route []route
|
||||
}
|
||||
|
||||
func (fr *fakeRouter) Handle(method, pattern string, handlers []macaron.Handler) *macaron.Route {
|
||||
fr.route = append(fr.route, route{
|
||||
pattern: pattern,
|
||||
method: method,
|
||||
handlers: handlers,
|
||||
})
|
||||
|
||||
return &macaron.Route{}
|
||||
}
|
||||
|
||||
func emptyHandlers(n int) []macaron.Handler {
|
||||
res := []macaron.Handler{}
|
||||
for i := 1; n >= i; i++ {
|
||||
res = append(res, emptyHandler(strconv.Itoa(i)))
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
func emptyHandler(name string) macaron.Handler {
|
||||
return struct{ name string }{name: name}
|
||||
}
|
||||
|
||||
func TestRouteSimpleRegister(t *testing.T) {
|
||||
testTable := []route{
|
||||
{method: "DELETE", pattern: "/admin", handlers: emptyHandlers(2)},
|
||||
{method: "GET", pattern: "/down", handlers: emptyHandlers(3)},
|
||||
}
|
||||
|
||||
// Setup
|
||||
rr := newRouteRegister(func(name string) macaron.Handler {
|
||||
return emptyHandler(name)
|
||||
})
|
||||
|
||||
rr.Delete("/admin", emptyHandler("1"))
|
||||
rr.Get("/down", emptyHandler("1"), emptyHandler("2"))
|
||||
|
||||
fr := &fakeRouter{}
|
||||
rr.Register(fr)
|
||||
|
||||
// Validation
|
||||
if len(fr.route) != len(testTable) {
|
||||
t.Errorf("want %v routes, got %v", len(testTable), len(fr.route))
|
||||
}
|
||||
|
||||
for i := range testTable {
|
||||
if testTable[i].method != fr.route[i].method {
|
||||
t.Errorf("want %s got %v", testTable[i].method, fr.route[i].method)
|
||||
}
|
||||
|
||||
if testTable[i].pattern != fr.route[i].pattern {
|
||||
t.Errorf("want %s got %v", testTable[i].pattern, fr.route[i].pattern)
|
||||
}
|
||||
|
||||
if len(testTable[i].handlers) != len(fr.route[i].handlers) {
|
||||
t.Errorf("want %d handlers got %d handlers \ntestcase: %v\nroute: %v\n",
|
||||
len(testTable[i].handlers),
|
||||
len(fr.route[i].handlers),
|
||||
testTable[i],
|
||||
fr.route[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRouteGroupedRegister(t *testing.T) {
|
||||
testTable := []route{
|
||||
{method: "DELETE", pattern: "/admin", handlers: emptyHandlers(1)},
|
||||
{method: "GET", pattern: "/down", handlers: emptyHandlers(2)},
|
||||
{method: "POST", pattern: "/user", handlers: emptyHandlers(1)},
|
||||
{method: "PUT", pattern: "/user/friends", handlers: emptyHandlers(1)},
|
||||
{method: "DELETE", pattern: "/user/admin", handlers: emptyHandlers(2)},
|
||||
{method: "GET", pattern: "/user/admin/all", handlers: emptyHandlers(4)},
|
||||
}
|
||||
|
||||
// Setup
|
||||
rr := newRouteRegister()
|
||||
|
||||
rr.Delete("/admin", emptyHandler("1"))
|
||||
rr.Get("/down", emptyHandler("1"), emptyHandler("2"))
|
||||
|
||||
rr.Group("/user", func(user RouteRegister) {
|
||||
user.Post("", emptyHandler("1"))
|
||||
user.Put("/friends", emptyHandler("2"))
|
||||
|
||||
user.Group("/admin", func(admin RouteRegister) {
|
||||
admin.Delete("", emptyHandler("3"))
|
||||
admin.Get("/all", emptyHandler("3"), emptyHandler("4"), emptyHandler("5"))
|
||||
|
||||
}, emptyHandler("3"))
|
||||
})
|
||||
|
||||
fr := &fakeRouter{}
|
||||
rr.Register(fr)
|
||||
|
||||
// Validation
|
||||
if len(fr.route) != len(testTable) {
|
||||
t.Errorf("want %v routes, got %v", len(testTable), len(fr.route))
|
||||
}
|
||||
|
||||
for i := range testTable {
|
||||
if testTable[i].method != fr.route[i].method {
|
||||
t.Errorf("want %s got %v", testTable[i].method, fr.route[i].method)
|
||||
}
|
||||
|
||||
if testTable[i].pattern != fr.route[i].pattern {
|
||||
t.Errorf("want %s got %v", testTable[i].pattern, fr.route[i].pattern)
|
||||
}
|
||||
|
||||
if len(testTable[i].handlers) != len(fr.route[i].handlers) {
|
||||
t.Errorf("want %d handlers got %d handlers \ntestcase: %v\nroute: %v\n",
|
||||
len(testTable[i].handlers),
|
||||
len(fr.route[i].handlers),
|
||||
testTable[i],
|
||||
fr.route[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNamedMiddlewareRouteRegister(t *testing.T) {
|
||||
testTable := []route{
|
||||
{method: "DELETE", pattern: "/admin", handlers: emptyHandlers(2)},
|
||||
{method: "GET", pattern: "/down", handlers: emptyHandlers(3)},
|
||||
{method: "POST", pattern: "/user", handlers: emptyHandlers(2)},
|
||||
{method: "PUT", pattern: "/user/friends", handlers: emptyHandlers(2)},
|
||||
{method: "DELETE", pattern: "/user/admin", handlers: emptyHandlers(3)},
|
||||
{method: "GET", pattern: "/user/admin/all", handlers: emptyHandlers(5)},
|
||||
}
|
||||
|
||||
// Setup
|
||||
rr := newRouteRegister(func(name string) macaron.Handler {
|
||||
return emptyHandler(name)
|
||||
})
|
||||
|
||||
rr.Delete("/admin", emptyHandler("1"))
|
||||
rr.Get("/down", emptyHandler("1"), emptyHandler("2"))
|
||||
|
||||
rr.Group("/user", func(user RouteRegister) {
|
||||
user.Post("", emptyHandler("1"))
|
||||
user.Put("/friends", emptyHandler("2"))
|
||||
|
||||
user.Group("/admin", func(admin RouteRegister) {
|
||||
admin.Delete("", emptyHandler("3"))
|
||||
admin.Get("/all", emptyHandler("3"), emptyHandler("4"), emptyHandler("5"))
|
||||
|
||||
}, emptyHandler("3"))
|
||||
})
|
||||
|
||||
fr := &fakeRouter{}
|
||||
rr.Register(fr)
|
||||
|
||||
// Validation
|
||||
if len(fr.route) != len(testTable) {
|
||||
t.Errorf("want %v routes, got %v", len(testTable), len(fr.route))
|
||||
}
|
||||
|
||||
for i := range testTable {
|
||||
if testTable[i].method != fr.route[i].method {
|
||||
t.Errorf("want %s got %v", testTable[i].method, fr.route[i].method)
|
||||
}
|
||||
|
||||
if testTable[i].pattern != fr.route[i].pattern {
|
||||
t.Errorf("want %s got %v", testTable[i].pattern, fr.route[i].pattern)
|
||||
}
|
||||
|
||||
if len(testTable[i].handlers) != len(fr.route[i].handlers) {
|
||||
t.Errorf("want %d handlers got %d handlers \ntestcase: %v\nroute: %v\n",
|
||||
len(testTable[i].handlers),
|
||||
len(fr.route[i].handlers),
|
||||
testTable[i],
|
||||
fr.route[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
+2
-2
@@ -47,7 +47,7 @@ func SignUp(c *middleware.Context, form dtos.SignUpForm) Response {
|
||||
Code: cmd.Code,
|
||||
})
|
||||
|
||||
metrics.M_Api_User_SignUpStarted.Inc(1)
|
||||
metrics.M_Api_User_SignUpStarted.Inc()
|
||||
|
||||
return Json(200, util.DynMap{"status": "SignUpCreated"})
|
||||
}
|
||||
@@ -111,7 +111,7 @@ func SignUpStep2(c *middleware.Context, form dtos.SignUpStep2Form) Response {
|
||||
}
|
||||
|
||||
loginUserWithUser(user, c)
|
||||
metrics.M_Api_User_SignUpCompleted.Inc(1)
|
||||
metrics.M_Api_User_SignUpCompleted.Inc()
|
||||
|
||||
return Json(200, apiResponse)
|
||||
}
|
||||
|
||||
@@ -54,7 +54,7 @@ func (g *GrafanaServerImpl) Start() {
|
||||
g.writePIDFile()
|
||||
|
||||
initSql()
|
||||
metrics.Init()
|
||||
metrics.Init(setting.Cfg)
|
||||
search.Init()
|
||||
login.Init()
|
||||
social.NewOAuthService()
|
||||
|
||||
@@ -1,122 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"math"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
// EWMAs continuously calculate an exponentially-weighted moving average
|
||||
// based on an outside source of clock ticks.
|
||||
type EWMA interface {
|
||||
Rate() float64
|
||||
Snapshot() EWMA
|
||||
Tick()
|
||||
Update(int64)
|
||||
}
|
||||
|
||||
// NewEWMA constructs a new EWMA with the given alpha.
|
||||
func NewEWMA(alpha float64) EWMA {
|
||||
if UseNilMetrics {
|
||||
return NilEWMA{}
|
||||
}
|
||||
return &StandardEWMA{alpha: alpha}
|
||||
}
|
||||
|
||||
// NewEWMA1 constructs a new EWMA for a one-minute moving average.
|
||||
func NewEWMA1() EWMA {
|
||||
return NewEWMA(1 - math.Exp(-5.0/60.0/1))
|
||||
}
|
||||
|
||||
// NewEWMA5 constructs a new EWMA for a five-minute moving average.
|
||||
func NewEWMA5() EWMA {
|
||||
return NewEWMA(1 - math.Exp(-5.0/60.0/5))
|
||||
}
|
||||
|
||||
// NewEWMA15 constructs a new EWMA for a fifteen-minute moving average.
|
||||
func NewEWMA15() EWMA {
|
||||
return NewEWMA(1 - math.Exp(-5.0/60.0/15))
|
||||
}
|
||||
|
||||
// EWMASnapshot is a read-only copy of another EWMA.
|
||||
type EWMASnapshot float64
|
||||
|
||||
// Rate returns the rate of events per second at the time the snapshot was
|
||||
// taken.
|
||||
func (a EWMASnapshot) Rate() float64 { return float64(a) }
|
||||
|
||||
// Snapshot returns the snapshot.
|
||||
func (a EWMASnapshot) Snapshot() EWMA { return a }
|
||||
|
||||
// Tick panics.
|
||||
func (EWMASnapshot) Tick() {
|
||||
panic("Tick called on an EWMASnapshot")
|
||||
}
|
||||
|
||||
// Update panics.
|
||||
func (EWMASnapshot) Update(int64) {
|
||||
panic("Update called on an EWMASnapshot")
|
||||
}
|
||||
|
||||
// NilEWMA is a no-op EWMA.
|
||||
type NilEWMA struct{}
|
||||
|
||||
// Rate is a no-op.
|
||||
func (NilEWMA) Rate() float64 { return 0.0 }
|
||||
|
||||
// Snapshot is a no-op.
|
||||
func (NilEWMA) Snapshot() EWMA { return NilEWMA{} }
|
||||
|
||||
// Tick is a no-op.
|
||||
func (NilEWMA) Tick() {}
|
||||
|
||||
// Update is a no-op.
|
||||
func (NilEWMA) Update(n int64) {}
|
||||
|
||||
// StandardEWMA is the standard implementation of an EWMA and tracks the number
|
||||
// of uncounted events and processes them on each tick. It uses the
|
||||
// sync/atomic package to manage uncounted events.
|
||||
type StandardEWMA struct {
|
||||
uncounted int64 // /!\ this should be the first member to ensure 64-bit alignment
|
||||
alpha float64
|
||||
rate float64
|
||||
init bool
|
||||
mutex sync.Mutex
|
||||
}
|
||||
|
||||
// Rate returns the moving average rate of events per second.
|
||||
func (a *StandardEWMA) Rate() float64 {
|
||||
a.mutex.Lock()
|
||||
defer a.mutex.Unlock()
|
||||
return a.rate * float64(1e9)
|
||||
}
|
||||
|
||||
// Snapshot returns a read-only copy of the EWMA.
|
||||
func (a *StandardEWMA) Snapshot() EWMA {
|
||||
return EWMASnapshot(a.Rate())
|
||||
}
|
||||
|
||||
// Tick ticks the clock to update the moving average. It assumes it is called
|
||||
// every five seconds.
|
||||
func (a *StandardEWMA) Tick() {
|
||||
count := atomic.LoadInt64(&a.uncounted)
|
||||
atomic.AddInt64(&a.uncounted, -count)
|
||||
instantRate := float64(count) / float64(5e9)
|
||||
a.mutex.Lock()
|
||||
defer a.mutex.Unlock()
|
||||
if a.init {
|
||||
a.rate += a.alpha * (instantRate - a.rate)
|
||||
} else {
|
||||
a.init = true
|
||||
a.rate = instantRate
|
||||
}
|
||||
}
|
||||
|
||||
// Update adds n uncounted events.
|
||||
func (a *StandardEWMA) Update(n int64) {
|
||||
atomic.AddInt64(&a.uncounted, n)
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
package metrics
|
||||
|
||||
// type comboCounterRef struct {
|
||||
// *MetricMeta
|
||||
// usageCounter Counter
|
||||
// metricCounter Counter
|
||||
// }
|
||||
//
|
||||
// func RegComboCounter(name string, tagStrings ...string) Counter {
|
||||
// meta := NewMetricMeta(name, tagStrings)
|
||||
// cr := &comboCounterRef{
|
||||
// MetricMeta: meta,
|
||||
// usageCounter: NewCounter(meta),
|
||||
// metricCounter: NewCounter(meta),
|
||||
// }
|
||||
//
|
||||
// UsageStats.Register(cr.usageCounter)
|
||||
// MetricStats.Register(cr.metricCounter)
|
||||
//
|
||||
// return cr
|
||||
// }
|
||||
//
|
||||
// func (c comboCounterRef) Clear() {
|
||||
// c.usageCounter.Clear()
|
||||
// c.metricCounter.Clear()
|
||||
// }
|
||||
//
|
||||
// func (c comboCounterRef) Count() int64 {
|
||||
// panic("Count called on a combocounter ref")
|
||||
// }
|
||||
//
|
||||
// // Dec panics.
|
||||
// func (c comboCounterRef) Dec(i int64) {
|
||||
// c.usageCounter.Dec(i)
|
||||
// c.metricCounter.Dec(i)
|
||||
// }
|
||||
//
|
||||
// // Inc panics.
|
||||
// func (c comboCounterRef) Inc(i int64) {
|
||||
// c.usageCounter.Inc(i)
|
||||
// c.metricCounter.Inc(i)
|
||||
// }
|
||||
//
|
||||
// func (c comboCounterRef) Snapshot() Metric {
|
||||
// return c.metricCounter.Snapshot()
|
||||
// }
|
||||
@@ -1,61 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import "github.com/grafana/grafana/pkg/log"
|
||||
|
||||
type MetricMeta struct {
|
||||
tags map[string]string
|
||||
name string
|
||||
}
|
||||
|
||||
func NewMetricMeta(name string, tagStrings []string) *MetricMeta {
|
||||
if len(tagStrings)%2 != 0 {
|
||||
log.Fatal(3, "Metrics: tags array is missing value for key, %v", tagStrings)
|
||||
}
|
||||
|
||||
tags := make(map[string]string)
|
||||
for i := 0; i < len(tagStrings); i += 2 {
|
||||
tags[tagStrings[i]] = tagStrings[i+1]
|
||||
}
|
||||
|
||||
return &MetricMeta{
|
||||
tags: tags,
|
||||
name: name,
|
||||
}
|
||||
}
|
||||
|
||||
func (m *MetricMeta) Name() string {
|
||||
return m.name
|
||||
}
|
||||
|
||||
func (m *MetricMeta) GetTagsCopy() map[string]string {
|
||||
if len(m.tags) == 0 {
|
||||
return make(map[string]string)
|
||||
}
|
||||
|
||||
copy := make(map[string]string)
|
||||
for k2, v2 := range m.tags {
|
||||
copy[k2] = v2
|
||||
}
|
||||
|
||||
return copy
|
||||
}
|
||||
|
||||
func (m *MetricMeta) StringifyTags() string {
|
||||
if len(m.tags) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
str := ""
|
||||
for key, value := range m.tags {
|
||||
str += "." + key + "_" + value
|
||||
}
|
||||
|
||||
return str
|
||||
}
|
||||
|
||||
type Metric interface {
|
||||
Name() string
|
||||
GetTagsCopy() map[string]string
|
||||
StringifyTags() string
|
||||
Snapshot() Metric
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import "sync/atomic"
|
||||
|
||||
// Counters hold an int64 value that can be incremented and decremented.
|
||||
type Counter interface {
|
||||
Metric
|
||||
|
||||
Clear()
|
||||
Count() int64
|
||||
Dec(int64)
|
||||
Inc(int64)
|
||||
}
|
||||
|
||||
// NewCounter constructs a new StandardCounter.
|
||||
func NewCounter(meta *MetricMeta) Counter {
|
||||
return &StandardCounter{
|
||||
MetricMeta: meta,
|
||||
count: 0,
|
||||
}
|
||||
}
|
||||
|
||||
func RegCounter(name string, tagStrings ...string) Counter {
|
||||
cr := NewCounter(NewMetricMeta(name, tagStrings))
|
||||
MetricStats.Register(cr)
|
||||
return cr
|
||||
}
|
||||
|
||||
// StandardCounter is the standard implementation of a Counter and uses the
|
||||
// sync/atomic package to manage a single int64 value.
|
||||
type StandardCounter struct {
|
||||
count int64 //Due to a bug in golang the 64bit variable need to come first to be 64bit aligned. https://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||
*MetricMeta
|
||||
}
|
||||
|
||||
// Clear sets the counter to zero.
|
||||
func (c *StandardCounter) Clear() {
|
||||
atomic.StoreInt64(&c.count, 0)
|
||||
}
|
||||
|
||||
// Count returns the current count.
|
||||
func (c *StandardCounter) Count() int64 {
|
||||
return atomic.LoadInt64(&c.count)
|
||||
}
|
||||
|
||||
// Dec decrements the counter by the given amount.
|
||||
func (c *StandardCounter) Dec(i int64) {
|
||||
atomic.AddInt64(&c.count, -i)
|
||||
}
|
||||
|
||||
// Inc increments the counter by the given amount.
|
||||
func (c *StandardCounter) Inc(i int64) {
|
||||
atomic.AddInt64(&c.count, i)
|
||||
}
|
||||
|
||||
func (c *StandardCounter) Snapshot() Metric {
|
||||
return &StandardCounter{
|
||||
MetricMeta: c.MetricMeta,
|
||||
count: c.count,
|
||||
}
|
||||
}
|
||||
@@ -1,11 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import "math"
|
||||
|
||||
func calculateDelta(oldValue, newValue int64) int64 {
|
||||
if oldValue < newValue {
|
||||
return newValue - oldValue
|
||||
} else {
|
||||
return (math.MaxInt64 - oldValue) + (newValue - math.MinInt64) + 1
|
||||
}
|
||||
}
|
||||
@@ -1,83 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import "sync/atomic"
|
||||
|
||||
// Gauges hold an int64 value that can be set arbitrarily.
|
||||
type Gauge interface {
|
||||
Metric
|
||||
|
||||
Update(int64)
|
||||
Value() int64
|
||||
}
|
||||
|
||||
func NewGauge(meta *MetricMeta) Gauge {
|
||||
if UseNilMetrics {
|
||||
return NilGauge{}
|
||||
}
|
||||
return &StandardGauge{
|
||||
MetricMeta: meta,
|
||||
value: 0,
|
||||
}
|
||||
}
|
||||
|
||||
func RegGauge(name string, tagStrings ...string) Gauge {
|
||||
tr := NewGauge(NewMetricMeta(name, tagStrings))
|
||||
MetricStats.Register(tr)
|
||||
return tr
|
||||
}
|
||||
|
||||
// GaugeSnapshot is a read-only copy of another Gauge.
|
||||
type GaugeSnapshot struct {
|
||||
value int64
|
||||
*MetricMeta
|
||||
}
|
||||
|
||||
// Snapshot returns the snapshot.
|
||||
func (g GaugeSnapshot) Snapshot() Metric { return g }
|
||||
|
||||
// Update panics.
|
||||
func (GaugeSnapshot) Update(int64) {
|
||||
panic("Update called on a GaugeSnapshot")
|
||||
}
|
||||
|
||||
// Value returns the value at the time the snapshot was taken.
|
||||
func (g GaugeSnapshot) Value() int64 { return g.value }
|
||||
|
||||
// NilGauge is a no-op Gauge.
|
||||
type NilGauge struct{ *MetricMeta }
|
||||
|
||||
// Snapshot is a no-op.
|
||||
func (NilGauge) Snapshot() Metric { return NilGauge{} }
|
||||
|
||||
// Update is a no-op.
|
||||
func (NilGauge) Update(v int64) {}
|
||||
|
||||
// Value is a no-op.
|
||||
func (NilGauge) Value() int64 { return 0 }
|
||||
|
||||
// StandardGauge is the standard implementation of a Gauge and uses the
|
||||
// sync/atomic package to manage a single int64 value.
|
||||
// atomic needs 64-bit aligned memory which is ensure for first word
|
||||
type StandardGauge struct {
|
||||
value int64
|
||||
*MetricMeta
|
||||
}
|
||||
|
||||
// Snapshot returns a read-only copy of the gauge.
|
||||
func (g *StandardGauge) Snapshot() Metric {
|
||||
return GaugeSnapshot{MetricMeta: g.MetricMeta, value: g.value}
|
||||
}
|
||||
|
||||
// Update updates the gauge's value.
|
||||
func (g *StandardGauge) Update(v int64) {
|
||||
atomic.StoreInt64(&g.value, v)
|
||||
}
|
||||
|
||||
// Value returns the gauge's current value.
|
||||
func (g *StandardGauge) Value() int64 {
|
||||
return atomic.LoadInt64(&g.value)
|
||||
}
|
||||
@@ -1,107 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"net"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
)
|
||||
|
||||
type GraphitePublisher struct {
|
||||
address string
|
||||
protocol string
|
||||
prefix string
|
||||
prevCounts map[string]int64
|
||||
}
|
||||
|
||||
func CreateGraphitePublisher() (*GraphitePublisher, error) {
|
||||
graphiteSection, err := setting.Cfg.GetSection("metrics.graphite")
|
||||
if err != nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
address := graphiteSection.Key("address").String()
|
||||
if address == "" {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
publisher := &GraphitePublisher{}
|
||||
publisher.prevCounts = make(map[string]int64)
|
||||
publisher.protocol = "tcp"
|
||||
publisher.prefix = graphiteSection.Key("prefix").MustString("prod.grafana.%(instance_name)s")
|
||||
publisher.address = address
|
||||
|
||||
safeInstanceName := strings.Replace(setting.InstanceName, ".", "_", -1)
|
||||
prefix := graphiteSection.Key("prefix").Value()
|
||||
|
||||
if prefix == "" {
|
||||
prefix = "prod.grafana.%(instance_name)s."
|
||||
}
|
||||
|
||||
publisher.prefix = strings.Replace(prefix, "%(instance_name)s", safeInstanceName, -1)
|
||||
return publisher, nil
|
||||
}
|
||||
|
||||
func (this *GraphitePublisher) Publish(metrics []Metric) {
|
||||
conn, err := net.DialTimeout(this.protocol, this.address, time.Second*5)
|
||||
|
||||
if err != nil {
|
||||
log.Error(3, "Metrics: GraphitePublisher: Failed to connect to %s!", err)
|
||||
return
|
||||
}
|
||||
|
||||
buf := bytes.NewBufferString("")
|
||||
now := time.Now().Unix()
|
||||
|
||||
for _, m := range metrics {
|
||||
metricName := this.prefix + m.Name() + m.StringifyTags()
|
||||
|
||||
switch metric := m.(type) {
|
||||
case Counter:
|
||||
this.addCount(buf, metricName+".count", metric.Count(), now)
|
||||
case Gauge:
|
||||
this.addCount(buf, metricName, metric.Value(), now)
|
||||
case Timer:
|
||||
percentiles := metric.Percentiles([]float64{0.25, 0.75, 0.90, 0.99})
|
||||
this.addCount(buf, metricName+".count", metric.Count(), now)
|
||||
this.addInt(buf, metricName+".max", metric.Max(), now)
|
||||
this.addInt(buf, metricName+".min", metric.Min(), now)
|
||||
this.addFloat(buf, metricName+".mean", metric.Mean(), now)
|
||||
this.addFloat(buf, metricName+".std", metric.StdDev(), now)
|
||||
this.addFloat(buf, metricName+".p25", percentiles[0], now)
|
||||
this.addFloat(buf, metricName+".p75", percentiles[1], now)
|
||||
this.addFloat(buf, metricName+".p90", percentiles[2], now)
|
||||
this.addFloat(buf, metricName+".p99", percentiles[3], now)
|
||||
}
|
||||
}
|
||||
|
||||
log.Trace("Metrics: GraphitePublisher.Publish() \n%s", buf)
|
||||
_, err = conn.Write(buf.Bytes())
|
||||
|
||||
if err != nil {
|
||||
log.Error(3, "Metrics: GraphitePublisher: Failed to send metrics! %s", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (this *GraphitePublisher) addInt(buf *bytes.Buffer, metric string, value int64, now int64) {
|
||||
buf.WriteString(fmt.Sprintf("%s %d %d\n", metric, value, now))
|
||||
}
|
||||
|
||||
func (this *GraphitePublisher) addFloat(buf *bytes.Buffer, metric string, value float64, now int64) {
|
||||
buf.WriteString(fmt.Sprintf("%s %f %d\n", metric, value, now))
|
||||
}
|
||||
|
||||
func (this *GraphitePublisher) addCount(buf *bytes.Buffer, metric string, value int64, now int64) {
|
||||
delta := value
|
||||
|
||||
if last, ok := this.prevCounts[metric]; ok {
|
||||
delta = calculateDelta(last, value)
|
||||
}
|
||||
|
||||
this.prevCounts[metric] = value
|
||||
buf.WriteString(fmt.Sprintf("%s %d %d\n", metric, delta, now))
|
||||
}
|
||||
@@ -1,77 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
|
||||
. "github.com/smartystreets/goconvey/convey"
|
||||
)
|
||||
|
||||
func TestGraphitePublisher(t *testing.T) {
|
||||
|
||||
setting.CustomInitPath = "conf/does_not_exist.ini"
|
||||
|
||||
Convey("Test graphite prefix replacement", t, func() {
|
||||
var err error
|
||||
err = setting.NewConfigContext(&setting.CommandLineArgs{
|
||||
HomePath: "../../",
|
||||
})
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
|
||||
sec, err := setting.Cfg.NewSection("metrics.graphite")
|
||||
sec.NewKey("prefix", "prod.grafana.%(instance_name)s.")
|
||||
sec.NewKey("address", "localhost:2001")
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
|
||||
setting.InstanceName = "hostname.with.dots.com"
|
||||
publisher, err := CreateGraphitePublisher()
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
So(publisher, ShouldNotBeNil)
|
||||
|
||||
So(publisher.prefix, ShouldEqual, "prod.grafana.hostname_with_dots_com.")
|
||||
So(publisher.address, ShouldEqual, "localhost:2001")
|
||||
})
|
||||
|
||||
Convey("Test graphite publisher default prefix", t, func() {
|
||||
var err error
|
||||
err = setting.NewConfigContext(&setting.CommandLineArgs{
|
||||
HomePath: "../../",
|
||||
})
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
|
||||
sec, err := setting.Cfg.NewSection("metrics.graphite")
|
||||
sec.NewKey("address", "localhost:2001")
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
|
||||
setting.InstanceName = "hostname.with.dots.com"
|
||||
publisher, err := CreateGraphitePublisher()
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
So(publisher, ShouldNotBeNil)
|
||||
|
||||
So(publisher.prefix, ShouldEqual, "prod.grafana.hostname_with_dots_com.")
|
||||
So(publisher.address, ShouldEqual, "localhost:2001")
|
||||
})
|
||||
|
||||
Convey("Test graphite publisher default values", t, func() {
|
||||
var err error
|
||||
err = setting.NewConfigContext(&setting.CommandLineArgs{
|
||||
HomePath: "../../",
|
||||
})
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
|
||||
_, err = setting.Cfg.NewSection("metrics.graphite")
|
||||
|
||||
publisher, err := CreateGraphitePublisher()
|
||||
|
||||
So(err, ShouldBeNil)
|
||||
So(publisher, ShouldBeNil)
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,396 @@
|
||||
// Copyright 2016 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Package graphite provides a bridge to push Prometheus metrics to a Graphite
|
||||
// server.
|
||||
package graphitebridge
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/common/expfmt"
|
||||
"github.com/prometheus/common/model"
|
||||
"golang.org/x/net/context"
|
||||
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultInterval = 15 * time.Second
|
||||
millisecondsPerSecond = 1000
|
||||
)
|
||||
|
||||
// HandlerErrorHandling defines how a Handler serving metrics will handle
|
||||
// errors.
|
||||
type HandlerErrorHandling int
|
||||
|
||||
// These constants cause handlers serving metrics to behave as described if
|
||||
// errors are encountered.
|
||||
const (
|
||||
// Ignore errors and try to push as many metrics to Graphite as possible.
|
||||
ContinueOnError HandlerErrorHandling = iota
|
||||
|
||||
// Abort the push to Graphite upon the first error encountered.
|
||||
AbortOnError
|
||||
)
|
||||
|
||||
var metricCategoryPrefix []string = []string{"proxy_", "api_", "page_", "alerting_", "aws_", "db_", "stat_", "go_", "process_"}
|
||||
var trimMetricPrefix []string = []string{"grafana_"}
|
||||
|
||||
// Config defines the Graphite bridge config.
|
||||
type Config struct {
|
||||
// The url to push data to. Required.
|
||||
URL string
|
||||
|
||||
// The prefix for the pushed Graphite metrics. Defaults to empty string.
|
||||
Prefix string
|
||||
|
||||
// The interval to use for pushing data to Graphite. Defaults to 15 seconds.
|
||||
Interval time.Duration
|
||||
|
||||
// The timeout for pushing metrics to Graphite. Defaults to 15 seconds.
|
||||
Timeout time.Duration
|
||||
|
||||
// The Gatherer to use for metrics. Defaults to prometheus.DefaultGatherer.
|
||||
Gatherer prometheus.Gatherer
|
||||
|
||||
// The logger that messages are written to. Defaults to no logging.
|
||||
Logger Logger
|
||||
|
||||
// ErrorHandling defines how errors are handled. Note that errors are
|
||||
// logged regardless of the configured ErrorHandling provided Logger
|
||||
// is not nil.
|
||||
ErrorHandling HandlerErrorHandling
|
||||
|
||||
// Graphite does not support ever increasing counter the same way
|
||||
// prometheus does. Rollups and ingestion might cannot handle ever
|
||||
// increasing counters. This option allows enabled the caller to
|
||||
// calculate the delta by saving the last sent counter in memory
|
||||
// and subtraction it from the collected value before sending.
|
||||
CountersAsDelta bool
|
||||
}
|
||||
|
||||
// Bridge pushes metrics to the configured Graphite server.
|
||||
type Bridge struct {
|
||||
url string
|
||||
prefix string
|
||||
countersAsDetlas bool
|
||||
interval time.Duration
|
||||
timeout time.Duration
|
||||
|
||||
errorHandling HandlerErrorHandling
|
||||
logger Logger
|
||||
|
||||
g prometheus.Gatherer
|
||||
|
||||
lastValue map[model.Fingerprint]float64
|
||||
}
|
||||
|
||||
// Logger is the minimal interface Bridge needs for logging. Note that
|
||||
// log.Logger from the standard library implements this interface, and it is
|
||||
// easy to implement by custom loggers, if they don't do so already anyway.
|
||||
type Logger interface {
|
||||
Println(v ...interface{})
|
||||
}
|
||||
|
||||
// NewBridge returns a pointer to a new Bridge struct.
|
||||
func NewBridge(c *Config) (*Bridge, error) {
|
||||
b := &Bridge{}
|
||||
|
||||
if c.URL == "" {
|
||||
return nil, errors.New("missing URL")
|
||||
}
|
||||
b.url = c.URL
|
||||
|
||||
if c.Gatherer == nil {
|
||||
b.g = prometheus.DefaultGatherer
|
||||
} else {
|
||||
b.g = c.Gatherer
|
||||
}
|
||||
|
||||
if c.Logger != nil {
|
||||
b.logger = c.Logger
|
||||
}
|
||||
|
||||
if c.Prefix != "" {
|
||||
b.prefix = c.Prefix
|
||||
}
|
||||
|
||||
var z time.Duration
|
||||
if c.Interval == z {
|
||||
b.interval = defaultInterval
|
||||
} else {
|
||||
b.interval = c.Interval
|
||||
}
|
||||
|
||||
if c.Timeout == z {
|
||||
b.timeout = defaultInterval
|
||||
} else {
|
||||
b.timeout = c.Timeout
|
||||
}
|
||||
|
||||
b.errorHandling = c.ErrorHandling
|
||||
b.lastValue = map[model.Fingerprint]float64{}
|
||||
b.countersAsDetlas = c.CountersAsDelta
|
||||
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// Run starts the event loop that pushes Prometheus metrics to Graphite at the
|
||||
// configured interval.
|
||||
func (b *Bridge) Run(ctx context.Context) {
|
||||
ticker := time.NewTicker(b.interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
if err := b.Push(); err != nil && b.logger != nil {
|
||||
b.logger.Println("error pushing to Graphite:", err)
|
||||
}
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Push pushes Prometheus metrics to the configured Graphite server.
|
||||
func (b *Bridge) Push() error {
|
||||
mfs, err := b.g.Gather()
|
||||
if err != nil || len(mfs) == 0 {
|
||||
switch b.errorHandling {
|
||||
case AbortOnError:
|
||||
return err
|
||||
case ContinueOnError:
|
||||
if b.logger != nil {
|
||||
b.logger.Println("continue on error:", err)
|
||||
}
|
||||
default:
|
||||
panic("unrecognized error handling value")
|
||||
}
|
||||
}
|
||||
|
||||
conn, err := net.DialTimeout("tcp", b.url, b.timeout)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
return b.writeMetrics(conn, mfs, b.prefix, model.Now())
|
||||
}
|
||||
|
||||
func (b *Bridge) writeMetrics(w io.Writer, mfs []*dto.MetricFamily, prefix string, now model.Time) error {
|
||||
for _, mf := range mfs {
|
||||
vec, err := expfmt.ExtractSamples(&expfmt.DecodeOptions{
|
||||
Timestamp: now,
|
||||
}, mf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
buf := bufio.NewWriter(w)
|
||||
for _, s := range vec {
|
||||
if err := writePrefix(buf, prefix); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := writeMetric(buf, s.Metric, mf); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
value := b.replaceCounterWithDelta(mf, s.Metric, s.Value)
|
||||
if _, err := fmt.Fprintf(buf, " %g %d\n", value, int64(s.Timestamp)/millisecondsPerSecond); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := buf.Flush(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeMetric(buf *bufio.Writer, m model.Metric, mf *dto.MetricFamily) error {
|
||||
metricName, hasName := m[model.MetricNameLabel]
|
||||
numLabels := len(m) - 1
|
||||
if !hasName {
|
||||
numLabels = len(m)
|
||||
}
|
||||
for _, v := range metricCategoryPrefix {
|
||||
if strings.HasPrefix(string(metricName), v) {
|
||||
group := strings.Replace(v, "_", " ", 1)
|
||||
metricName = model.LabelValue(strings.Replace(string(metricName), v, group, 1))
|
||||
}
|
||||
}
|
||||
|
||||
for _, v := range trimMetricPrefix {
|
||||
if strings.HasPrefix(string(metricName), v) {
|
||||
metricName = model.LabelValue(strings.Replace(string(metricName), v, "", 1))
|
||||
}
|
||||
}
|
||||
|
||||
labelStrings := make([]string, 0, numLabels)
|
||||
for label, value := range m {
|
||||
if label != model.MetricNameLabel {
|
||||
labelStrings = append(labelStrings, fmt.Sprintf("%s %s", string(label), string(value)))
|
||||
}
|
||||
}
|
||||
|
||||
var err error
|
||||
switch numLabels {
|
||||
case 0:
|
||||
if hasName {
|
||||
if err := writeSanitized(buf, string(metricName)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
default:
|
||||
sort.Strings(labelStrings)
|
||||
if err = writeSanitized(buf, string(metricName)); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, s := range labelStrings {
|
||||
if err = buf.WriteByte('.'); err != nil {
|
||||
return err
|
||||
}
|
||||
if err = writeSanitized(buf, s); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if err = addExtentionConventionForRollups(buf, mf, m); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func addExtentionConventionForRollups(buf *bufio.Writer, mf *dto.MetricFamily, m model.Metric) error {
|
||||
// Adding `.count` `.sum` suffix makes it possible to configure
|
||||
// different rollup strategies based on metric type
|
||||
|
||||
mfType := mf.GetType()
|
||||
var err error
|
||||
if mfType == dto.MetricType_COUNTER {
|
||||
if _, err = fmt.Fprint(buf, ".count"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if mfType == dto.MetricType_SUMMARY || mfType == dto.MetricType_HISTOGRAM {
|
||||
if strings.HasSuffix(string(m[model.MetricNameLabel]), "_count") {
|
||||
if _, err = fmt.Fprint(buf, ".count"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
if mfType == dto.MetricType_HISTOGRAM {
|
||||
if strings.HasSuffix(string(m[model.MetricNameLabel]), "_sum") {
|
||||
if _, err = fmt.Fprint(buf, ".sum"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func writePrefix(buf *bufio.Writer, s string) error {
|
||||
for _, c := range s {
|
||||
if _, err := buf.WriteRune(replaceInvalid(c)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeSanitized(buf *bufio.Writer, s string) error {
|
||||
prevUnderscore := false
|
||||
|
||||
for _, c := range s {
|
||||
c = replaceInvalidRune(c)
|
||||
if c == '_' {
|
||||
if prevUnderscore {
|
||||
continue
|
||||
}
|
||||
prevUnderscore = true
|
||||
} else {
|
||||
prevUnderscore = false
|
||||
}
|
||||
if _, err := buf.WriteRune(c); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func replaceInvalid(c rune) rune {
|
||||
if c == ' ' || c == '.' {
|
||||
return '.'
|
||||
}
|
||||
return replaceInvalidRune(c)
|
||||
}
|
||||
|
||||
func replaceInvalidRune(c rune) rune {
|
||||
if c == ' ' {
|
||||
return '.'
|
||||
}
|
||||
if !((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '_' || c == ':' || (c >= '0' && c <= '9')) {
|
||||
return '_'
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func (b *Bridge) replaceCounterWithDelta(mf *dto.MetricFamily, metric model.Metric, value model.SampleValue) float64 {
|
||||
if !b.countersAsDetlas {
|
||||
return float64(value)
|
||||
}
|
||||
|
||||
mfType := mf.GetType()
|
||||
if mfType == dto.MetricType_COUNTER {
|
||||
return b.returnDelta(metric, value)
|
||||
}
|
||||
|
||||
if mfType == dto.MetricType_SUMMARY {
|
||||
if strings.HasSuffix(string(metric[model.MetricNameLabel]), "_count") {
|
||||
return b.returnDelta(metric, value)
|
||||
}
|
||||
}
|
||||
|
||||
return float64(value)
|
||||
}
|
||||
|
||||
func (b *Bridge) returnDelta(metric model.Metric, value model.SampleValue) float64 {
|
||||
key := metric.Fingerprint()
|
||||
_, exists := b.lastValue[key]
|
||||
if !exists {
|
||||
b.lastValue[key] = 0
|
||||
}
|
||||
|
||||
delta := float64(value) - b.lastValue[key]
|
||||
b.lastValue[key] = float64(value)
|
||||
|
||||
return delta
|
||||
}
|
||||
@@ -0,0 +1,503 @@
|
||||
package graphitebridge
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"io"
|
||||
"net"
|
||||
"regexp"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
"github.com/prometheus/common/model"
|
||||
)
|
||||
|
||||
func TestCountersAsDelta(t *testing.T) {
|
||||
b, _ := NewBridge(&Config{
|
||||
URL: "localhost:12345",
|
||||
CountersAsDelta: true,
|
||||
})
|
||||
ty := dto.MetricType(0)
|
||||
mf := &dto.MetricFamily{
|
||||
Type: &ty,
|
||||
Metric: []*dto.Metric{},
|
||||
}
|
||||
m := model.Metric{}
|
||||
|
||||
var want float64
|
||||
var got float64
|
||||
want = float64(1)
|
||||
got = b.replaceCounterWithDelta(mf, m, model.SampleValue(1))
|
||||
if got != want {
|
||||
t.Fatalf("want %v got %v", want, got)
|
||||
}
|
||||
|
||||
got = b.replaceCounterWithDelta(mf, m, model.SampleValue(2))
|
||||
if got != want {
|
||||
t.Fatalf("want %v got %v", want, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCountersAsDeltaDisabled(t *testing.T) {
|
||||
b, _ := NewBridge(&Config{
|
||||
URL: "localhost:12345",
|
||||
CountersAsDelta: false,
|
||||
})
|
||||
ty := dto.MetricType(0)
|
||||
mf := &dto.MetricFamily{
|
||||
Type: &ty,
|
||||
Metric: []*dto.Metric{},
|
||||
}
|
||||
m := model.Metric{}
|
||||
|
||||
var want float64
|
||||
var got float64
|
||||
want = float64(1)
|
||||
got = b.replaceCounterWithDelta(mf, m, model.SampleValue(1))
|
||||
if got != want {
|
||||
t.Fatalf("want %v got %v", want, got)
|
||||
}
|
||||
|
||||
want = float64(2)
|
||||
got = b.replaceCounterWithDelta(mf, m, model.SampleValue(2))
|
||||
if got != want {
|
||||
t.Fatalf("want %v got %v", want, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSanitize(t *testing.T) {
|
||||
testCases := []struct {
|
||||
in, out string
|
||||
}{
|
||||
{in: "hello", out: "hello"},
|
||||
{in: "hE/l1o", out: "hE_l1o"},
|
||||
{in: "he,*ll(.o", out: "he_ll_o"},
|
||||
{in: "hello_there%^&", out: "hello_there_"},
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
w := bufio.NewWriter(&buf)
|
||||
|
||||
for i, tc := range testCases {
|
||||
if err := writeSanitized(w, tc.in); err != nil {
|
||||
t.Fatalf("write failed: %v", err)
|
||||
}
|
||||
if err := w.Flush(); err != nil {
|
||||
t.Fatalf("flush failed: %v", err)
|
||||
}
|
||||
|
||||
if want, got := tc.out, buf.String(); want != got {
|
||||
t.Fatalf("test case index %d: got sanitized string %s, want %s", i, got, want)
|
||||
}
|
||||
|
||||
buf.Reset()
|
||||
}
|
||||
}
|
||||
|
||||
func TestSanitizePrefix(t *testing.T) {
|
||||
testCases := []struct {
|
||||
in, out string
|
||||
}{
|
||||
{in: "service.prod.", out: "service.prod."},
|
||||
{in: "service.prod", out: "service.prod"},
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
w := bufio.NewWriter(&buf)
|
||||
|
||||
for i, tc := range testCases {
|
||||
if err := writePrefix(w, tc.in); err != nil {
|
||||
t.Fatalf("write failed: %v", err)
|
||||
}
|
||||
if err := w.Flush(); err != nil {
|
||||
t.Fatalf("flush failed: %v", err)
|
||||
}
|
||||
|
||||
if want, got := tc.out, buf.String(); want != got {
|
||||
t.Fatalf("test case index %d: got sanitized string %s, want %s", i, got, want)
|
||||
}
|
||||
|
||||
buf.Reset()
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteSummary(t *testing.T) {
|
||||
sumVec := prometheus.NewSummaryVec(
|
||||
prometheus.SummaryOpts{
|
||||
Name: "name",
|
||||
Help: "docstring",
|
||||
ConstLabels: prometheus.Labels{"constname": "constvalue"},
|
||||
Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},
|
||||
},
|
||||
[]string{"labelname"},
|
||||
)
|
||||
|
||||
reg := prometheus.NewRegistry()
|
||||
reg.MustRegister(sumVec)
|
||||
|
||||
b, err := NewBridge(&Config{
|
||||
URL: "localhost:8080",
|
||||
Gatherer: reg,
|
||||
CountersAsDelta: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("cannot create bridge. err: %v", err)
|
||||
}
|
||||
|
||||
sumVec.WithLabelValues("val1").Observe(float64(10))
|
||||
sumVec.WithLabelValues("val1").Observe(float64(20))
|
||||
sumVec.WithLabelValues("val1").Observe(float64(30))
|
||||
sumVec.WithLabelValues("val2").Observe(float64(20))
|
||||
sumVec.WithLabelValues("val2").Observe(float64(30))
|
||||
sumVec.WithLabelValues("val2").Observe(float64(40))
|
||||
|
||||
mfs, err := reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
now := model.Time(1477043083)
|
||||
var buf bytes.Buffer
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", now)
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want := `prefix.name.constname.constvalue.labelname.val1.quantile.0_5 20 1477043
|
||||
prefix.name.constname.constvalue.labelname.val1.quantile.0_9 30 1477043
|
||||
prefix.name.constname.constvalue.labelname.val1.quantile.0_99 30 1477043
|
||||
prefix.name_sum.constname.constvalue.labelname.val1 60 1477043
|
||||
prefix.name_count.constname.constvalue.labelname.val1.count 3 1477043
|
||||
prefix.name.constname.constvalue.labelname.val2.quantile.0_5 30 1477043
|
||||
prefix.name.constname.constvalue.labelname.val2.quantile.0_9 40 1477043
|
||||
prefix.name.constname.constvalue.labelname.val2.quantile.0_99 40 1477043
|
||||
prefix.name_sum.constname.constvalue.labelname.val2 90 1477043
|
||||
prefix.name_count.constname.constvalue.labelname.val2.count 3 1477043
|
||||
`
|
||||
|
||||
if got := buf.String(); want != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteHistogram(t *testing.T) {
|
||||
histVec := prometheus.NewHistogramVec(
|
||||
prometheus.HistogramOpts{
|
||||
Name: "name",
|
||||
Help: "docstring",
|
||||
ConstLabels: prometheus.Labels{"constname": "constvalue"},
|
||||
Buckets: []float64{0.01, 0.02, 0.05, 0.1},
|
||||
},
|
||||
[]string{"labelname"},
|
||||
)
|
||||
|
||||
reg := prometheus.NewRegistry()
|
||||
reg.MustRegister(histVec)
|
||||
|
||||
b, err := NewBridge(&Config{
|
||||
URL: "localhost:8080",
|
||||
Gatherer: reg,
|
||||
CountersAsDelta: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("error creating bridge: %v", err)
|
||||
}
|
||||
|
||||
histVec.WithLabelValues("val1").Observe(float64(10))
|
||||
histVec.WithLabelValues("val1").Observe(float64(20))
|
||||
histVec.WithLabelValues("val1").Observe(float64(30))
|
||||
histVec.WithLabelValues("val2").Observe(float64(20))
|
||||
histVec.WithLabelValues("val2").Observe(float64(30))
|
||||
histVec.WithLabelValues("val2").Observe(float64(40))
|
||||
|
||||
mfs, err := reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
now := model.Time(1477043083)
|
||||
var buf bytes.Buffer
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", now)
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want := `prefix.name_bucket.constname.constvalue.labelname.val1.le.0_01 0 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val1.le.0_02 0 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val1.le.0_05 0 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val1.le.0_1 0 1477043
|
||||
prefix.name_sum.constname.constvalue.labelname.val1.sum 60 1477043
|
||||
prefix.name_count.constname.constvalue.labelname.val1.count 3 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val1.le._Inf 3 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val2.le.0_01 0 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val2.le.0_02 0 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val2.le.0_05 0 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val2.le.0_1 0 1477043
|
||||
prefix.name_sum.constname.constvalue.labelname.val2.sum 90 1477043
|
||||
prefix.name_count.constname.constvalue.labelname.val2.count 3 1477043
|
||||
prefix.name_bucket.constname.constvalue.labelname.val2.le._Inf 3 1477043
|
||||
`
|
||||
if got := buf.String(); want != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCounterVec(t *testing.T) {
|
||||
cntVec := prometheus.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Name: "page_response",
|
||||
Help: "docstring",
|
||||
ConstLabels: prometheus.Labels{"constname": "constvalue"},
|
||||
},
|
||||
[]string{"labelname"},
|
||||
)
|
||||
|
||||
reg := prometheus.NewRegistry()
|
||||
reg.MustRegister(cntVec)
|
||||
|
||||
cntVec.WithLabelValues("val1").Inc()
|
||||
cntVec.WithLabelValues("val2").Inc()
|
||||
|
||||
b, err := NewBridge(&Config{
|
||||
URL: "localhost:8080",
|
||||
Gatherer: reg,
|
||||
CountersAsDelta: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("error creating bridge: %v", err)
|
||||
}
|
||||
|
||||
// first collect
|
||||
mfs, err := reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", model.Time(1477043083))
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want := `prefix.page.response.constname.constvalue.labelname.val1.count 1 1477043
|
||||
prefix.page.response.constname.constvalue.labelname.val2.count 1 1477043
|
||||
`
|
||||
if got := buf.String(); want != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want, got)
|
||||
}
|
||||
|
||||
//next collect
|
||||
cntVec.WithLabelValues("val1").Inc()
|
||||
cntVec.WithLabelValues("val2").Inc()
|
||||
|
||||
mfs, err = reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
buf = bytes.Buffer{}
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", model.Time(1477053083))
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want2 := `prefix.page.response.constname.constvalue.labelname.val1.count 1 1477053
|
||||
prefix.page.response.constname.constvalue.labelname.val2.count 1 1477053
|
||||
`
|
||||
if got := buf.String(); want2 != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want2, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCounter(t *testing.T) {
|
||||
cntVec := prometheus.NewCounter(
|
||||
prometheus.CounterOpts{
|
||||
Name: "page_response",
|
||||
Help: "docstring",
|
||||
ConstLabels: prometheus.Labels{"constname": "constvalue"},
|
||||
})
|
||||
|
||||
reg := prometheus.NewRegistry()
|
||||
reg.MustRegister(cntVec)
|
||||
|
||||
cntVec.Inc()
|
||||
|
||||
b, err := NewBridge(&Config{
|
||||
URL: "localhost:8080",
|
||||
Gatherer: reg,
|
||||
CountersAsDelta: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("error creating bridge: %v", err)
|
||||
}
|
||||
|
||||
// first collect
|
||||
mfs, err := reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", model.Time(1477043083))
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want := "prefix.page.response.constname.constvalue.count 1 1477043\n"
|
||||
if got := buf.String(); want != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want, got)
|
||||
}
|
||||
|
||||
//next collect
|
||||
cntVec.Inc()
|
||||
|
||||
mfs, err = reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
buf = bytes.Buffer{}
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", model.Time(1477053083))
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want2 := "prefix.page.response.constname.constvalue.count 1 1477053\n"
|
||||
if got := buf.String(); want2 != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want2, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTrimGrafanaNamespace(t *testing.T) {
|
||||
cntVec := prometheus.NewCounter(
|
||||
prometheus.CounterOpts{
|
||||
Name: "grafana_http_request_total",
|
||||
Help: "docstring",
|
||||
ConstLabels: prometheus.Labels{"constname": "constvalue"},
|
||||
})
|
||||
|
||||
reg := prometheus.NewRegistry()
|
||||
reg.MustRegister(cntVec)
|
||||
|
||||
cntVec.Inc()
|
||||
|
||||
b, err := NewBridge(&Config{
|
||||
URL: "localhost:8080",
|
||||
Gatherer: reg,
|
||||
CountersAsDelta: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("error creating bridge: %v", err)
|
||||
}
|
||||
|
||||
// first collect
|
||||
mfs, err := reg.Gather()
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
err = b.writeMetrics(&buf, mfs, "prefix.", model.Time(1477043083))
|
||||
if err != nil {
|
||||
t.Fatalf("error: %v", err)
|
||||
}
|
||||
|
||||
want := "prefix.http_request_total.constname.constvalue.count 1 1477043\n"
|
||||
if got := buf.String(); want != got {
|
||||
t.Fatalf("wanted \n%s\n, got \n%s\n", want, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPush(t *testing.T) {
|
||||
reg := prometheus.NewRegistry()
|
||||
cntVec := prometheus.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Name: "name",
|
||||
Help: "docstring",
|
||||
ConstLabels: prometheus.Labels{"constname": "constvalue"},
|
||||
},
|
||||
[]string{"labelname"},
|
||||
)
|
||||
cntVec.WithLabelValues("val1").Inc()
|
||||
cntVec.WithLabelValues("val2").Inc()
|
||||
reg.MustRegister(cntVec)
|
||||
|
||||
host := "localhost"
|
||||
port := ":56789"
|
||||
b, err := NewBridge(&Config{
|
||||
URL: host + port,
|
||||
Gatherer: reg,
|
||||
Prefix: "prefix.",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("error creating bridge: %v", err)
|
||||
}
|
||||
|
||||
nmg, err := newMockGraphite(port)
|
||||
if err != nil {
|
||||
t.Fatalf("error creating mock graphite: %v", err)
|
||||
}
|
||||
defer nmg.Close()
|
||||
|
||||
err = b.Push()
|
||||
if err != nil {
|
||||
t.Fatalf("error pushing: %v", err)
|
||||
}
|
||||
|
||||
wants := []string{
|
||||
"prefix.name.constname.constvalue.labelname.val1.count 1",
|
||||
"prefix.name.constname.constvalue.labelname.val2.count 1",
|
||||
}
|
||||
|
||||
select {
|
||||
case got := <-nmg.readc:
|
||||
for _, want := range wants {
|
||||
matched, err := regexp.MatchString(want, got)
|
||||
if err != nil {
|
||||
t.Fatalf("error pushing: %v", err)
|
||||
}
|
||||
if !matched {
|
||||
t.Fatalf("missing metric:\nno match for %s received by server:\n%s", want, got)
|
||||
}
|
||||
}
|
||||
return
|
||||
case err := <-nmg.errc:
|
||||
t.Fatalf("error reading push: %v", err)
|
||||
case <-time.After(50 * time.Millisecond):
|
||||
t.Fatalf("no result from graphite server")
|
||||
}
|
||||
}
|
||||
|
||||
func newMockGraphite(port string) (*mockGraphite, error) {
|
||||
readc := make(chan string)
|
||||
errc := make(chan error)
|
||||
ln, err := net.Listen("tcp", port)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
go func() {
|
||||
conn, err := ln.Accept()
|
||||
if err != nil {
|
||||
errc <- err
|
||||
}
|
||||
var b bytes.Buffer
|
||||
io.Copy(&b, conn)
|
||||
readc <- b.String()
|
||||
}()
|
||||
|
||||
return &mockGraphite{
|
||||
readc: readc,
|
||||
errc: errc,
|
||||
Listener: ln,
|
||||
}, nil
|
||||
}
|
||||
|
||||
type mockGraphite struct {
|
||||
readc chan string
|
||||
errc chan error
|
||||
|
||||
net.Listener
|
||||
}
|
||||
@@ -1,189 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
// Histograms calculate distribution statistics from a series of int64 values.
|
||||
type Histogram interface {
|
||||
Metric
|
||||
|
||||
Clear()
|
||||
Count() int64
|
||||
Max() int64
|
||||
Mean() float64
|
||||
Min() int64
|
||||
Percentile(float64) float64
|
||||
Percentiles([]float64) []float64
|
||||
StdDev() float64
|
||||
Sum() int64
|
||||
Update(int64)
|
||||
Variance() float64
|
||||
}
|
||||
|
||||
func NewHistogram(meta *MetricMeta, s Sample) Histogram {
|
||||
return &StandardHistogram{
|
||||
MetricMeta: meta,
|
||||
sample: s,
|
||||
}
|
||||
}
|
||||
|
||||
// HistogramSnapshot is a read-only copy of another Histogram.
|
||||
type HistogramSnapshot struct {
|
||||
*MetricMeta
|
||||
sample *SampleSnapshot
|
||||
}
|
||||
|
||||
// Clear panics.
|
||||
func (*HistogramSnapshot) Clear() {
|
||||
panic("Clear called on a HistogramSnapshot")
|
||||
}
|
||||
|
||||
// Count returns the number of samples recorded at the time the snapshot was
|
||||
// taken.
|
||||
func (h *HistogramSnapshot) Count() int64 { return h.sample.Count() }
|
||||
|
||||
// Max returns the maximum value in the sample at the time the snapshot was
|
||||
// taken.
|
||||
func (h *HistogramSnapshot) Max() int64 { return h.sample.Max() }
|
||||
|
||||
// Mean returns the mean of the values in the sample at the time the snapshot
|
||||
// was taken.
|
||||
func (h *HistogramSnapshot) Mean() float64 { return h.sample.Mean() }
|
||||
|
||||
// Min returns the minimum value in the sample at the time the snapshot was
|
||||
// taken.
|
||||
func (h *HistogramSnapshot) Min() int64 { return h.sample.Min() }
|
||||
|
||||
// Percentile returns an arbitrary percentile of values in the sample at the
|
||||
// time the snapshot was taken.
|
||||
func (h *HistogramSnapshot) Percentile(p float64) float64 {
|
||||
return h.sample.Percentile(p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of values in the sample
|
||||
// at the time the snapshot was taken.
|
||||
func (h *HistogramSnapshot) Percentiles(ps []float64) []float64 {
|
||||
return h.sample.Percentiles(ps)
|
||||
}
|
||||
|
||||
// Sample returns the Sample underlying the histogram.
|
||||
func (h *HistogramSnapshot) Sample() Sample { return h.sample }
|
||||
|
||||
// Snapshot returns the snapshot.
|
||||
func (h *HistogramSnapshot) Snapshot() Metric { return h }
|
||||
|
||||
// StdDev returns the standard deviation of the values in the sample at the
|
||||
// time the snapshot was taken.
|
||||
func (h *HistogramSnapshot) StdDev() float64 { return h.sample.StdDev() }
|
||||
|
||||
// Sum returns the sum in the sample at the time the snapshot was taken.
|
||||
func (h *HistogramSnapshot) Sum() int64 { return h.sample.Sum() }
|
||||
|
||||
// Update panics.
|
||||
func (*HistogramSnapshot) Update(int64) {
|
||||
panic("Update called on a HistogramSnapshot")
|
||||
}
|
||||
|
||||
// Variance returns the variance of inputs at the time the snapshot was taken.
|
||||
func (h *HistogramSnapshot) Variance() float64 { return h.sample.Variance() }
|
||||
|
||||
// NilHistogram is a no-op Histogram.
|
||||
type NilHistogram struct {
|
||||
*MetricMeta
|
||||
}
|
||||
|
||||
// Clear is a no-op.
|
||||
func (NilHistogram) Clear() {}
|
||||
|
||||
// Count is a no-op.
|
||||
func (NilHistogram) Count() int64 { return 0 }
|
||||
|
||||
// Max is a no-op.
|
||||
func (NilHistogram) Max() int64 { return 0 }
|
||||
|
||||
// Mean is a no-op.
|
||||
func (NilHistogram) Mean() float64 { return 0.0 }
|
||||
|
||||
// Min is a no-op.
|
||||
func (NilHistogram) Min() int64 { return 0 }
|
||||
|
||||
// Percentile is a no-op.
|
||||
func (NilHistogram) Percentile(p float64) float64 { return 0.0 }
|
||||
|
||||
// Percentiles is a no-op.
|
||||
func (NilHistogram) Percentiles(ps []float64) []float64 {
|
||||
return make([]float64, len(ps))
|
||||
}
|
||||
|
||||
// Sample is a no-op.
|
||||
func (NilHistogram) Sample() Sample { return NilSample{} }
|
||||
|
||||
// Snapshot is a no-op.
|
||||
func (n NilHistogram) Snapshot() Metric { return n }
|
||||
|
||||
// StdDev is a no-op.
|
||||
func (NilHistogram) StdDev() float64 { return 0.0 }
|
||||
|
||||
// Sum is a no-op.
|
||||
func (NilHistogram) Sum() int64 { return 0 }
|
||||
|
||||
// Update is a no-op.
|
||||
func (NilHistogram) Update(v int64) {}
|
||||
|
||||
// Variance is a no-op.
|
||||
func (NilHistogram) Variance() float64 { return 0.0 }
|
||||
|
||||
// StandardHistogram is the standard implementation of a Histogram and uses a
|
||||
// Sample to bound its memory use.
|
||||
type StandardHistogram struct {
|
||||
*MetricMeta
|
||||
sample Sample
|
||||
}
|
||||
|
||||
// Clear clears the histogram and its sample.
|
||||
func (h *StandardHistogram) Clear() { h.sample.Clear() }
|
||||
|
||||
// Count returns the number of samples recorded since the histogram was last
|
||||
// cleared.
|
||||
func (h *StandardHistogram) Count() int64 { return h.sample.Count() }
|
||||
|
||||
// Max returns the maximum value in the sample.
|
||||
func (h *StandardHistogram) Max() int64 { return h.sample.Max() }
|
||||
|
||||
// Mean returns the mean of the values in the sample.
|
||||
func (h *StandardHistogram) Mean() float64 { return h.sample.Mean() }
|
||||
|
||||
// Min returns the minimum value in the sample.
|
||||
func (h *StandardHistogram) Min() int64 { return h.sample.Min() }
|
||||
|
||||
// Percentile returns an arbitrary percentile of the values in the sample.
|
||||
func (h *StandardHistogram) Percentile(p float64) float64 {
|
||||
return h.sample.Percentile(p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of the values in the
|
||||
// sample.
|
||||
func (h *StandardHistogram) Percentiles(ps []float64) []float64 {
|
||||
return h.sample.Percentiles(ps)
|
||||
}
|
||||
|
||||
// Sample returns the Sample underlying the histogram.
|
||||
func (h *StandardHistogram) Sample() Sample { return h.sample }
|
||||
|
||||
// Snapshot returns a read-only copy of the histogram.
|
||||
func (h *StandardHistogram) Snapshot() Metric {
|
||||
return &HistogramSnapshot{sample: h.sample.Snapshot().(*SampleSnapshot)}
|
||||
}
|
||||
|
||||
// StdDev returns the standard deviation of the values in the sample.
|
||||
func (h *StandardHistogram) StdDev() float64 { return h.sample.StdDev() }
|
||||
|
||||
// Sum returns the sum in the sample.
|
||||
func (h *StandardHistogram) Sum() int64 { return h.sample.Sum() }
|
||||
|
||||
// Update samples a new value.
|
||||
func (h *StandardHistogram) Update(v int64) { h.sample.Update(v) }
|
||||
|
||||
// Variance returns the variance of the values in the sample.
|
||||
func (h *StandardHistogram) Variance() float64 { return h.sample.Variance() }
|
||||
@@ -1,90 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import "testing"
|
||||
|
||||
func BenchmarkHistogram(b *testing.B) {
|
||||
h := NewHistogram(nil, NewUniformSample(100))
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
h.Update(int64(i))
|
||||
}
|
||||
}
|
||||
|
||||
func TestHistogram10000(t *testing.T) {
|
||||
h := NewHistogram(nil, NewUniformSample(100000))
|
||||
for i := 1; i <= 10000; i++ {
|
||||
h.Update(int64(i))
|
||||
}
|
||||
testHistogram10000(t, h)
|
||||
}
|
||||
|
||||
func TestHistogramEmpty(t *testing.T) {
|
||||
h := NewHistogram(nil, NewUniformSample(100))
|
||||
if count := h.Count(); 0 != count {
|
||||
t.Errorf("h.Count(): 0 != %v\n", count)
|
||||
}
|
||||
if min := h.Min(); 0 != min {
|
||||
t.Errorf("h.Min(): 0 != %v\n", min)
|
||||
}
|
||||
if max := h.Max(); 0 != max {
|
||||
t.Errorf("h.Max(): 0 != %v\n", max)
|
||||
}
|
||||
if mean := h.Mean(); 0.0 != mean {
|
||||
t.Errorf("h.Mean(): 0.0 != %v\n", mean)
|
||||
}
|
||||
if stdDev := h.StdDev(); 0.0 != stdDev {
|
||||
t.Errorf("h.StdDev(): 0.0 != %v\n", stdDev)
|
||||
}
|
||||
ps := h.Percentiles([]float64{0.5, 0.75, 0.99})
|
||||
if 0.0 != ps[0] {
|
||||
t.Errorf("median: 0.0 != %v\n", ps[0])
|
||||
}
|
||||
if 0.0 != ps[1] {
|
||||
t.Errorf("75th percentile: 0.0 != %v\n", ps[1])
|
||||
}
|
||||
if 0.0 != ps[2] {
|
||||
t.Errorf("99th percentile: 0.0 != %v\n", ps[2])
|
||||
}
|
||||
}
|
||||
|
||||
func TestHistogramSnapshot(t *testing.T) {
|
||||
h := NewHistogram(nil, NewUniformSample(100000))
|
||||
for i := 1; i <= 10000; i++ {
|
||||
h.Update(int64(i))
|
||||
}
|
||||
snapshot := h.Snapshot().(Histogram)
|
||||
h.Update(0)
|
||||
testHistogram10000(t, snapshot)
|
||||
}
|
||||
|
||||
func testHistogram10000(t *testing.T, h Histogram) {
|
||||
if count := h.Count(); 10000 != count {
|
||||
t.Errorf("h.Count(): 10000 != %v\n", count)
|
||||
}
|
||||
if min := h.Min(); 1 != min {
|
||||
t.Errorf("h.Min(): 1 != %v\n", min)
|
||||
}
|
||||
if max := h.Max(); 10000 != max {
|
||||
t.Errorf("h.Max(): 10000 != %v\n", max)
|
||||
}
|
||||
if mean := h.Mean(); 5000.5 != mean {
|
||||
t.Errorf("h.Mean(): 5000.5 != %v\n", mean)
|
||||
}
|
||||
if stdDev := h.StdDev(); 2886.751331514372 != stdDev {
|
||||
t.Errorf("h.StdDev(): 2886.751331514372 != %v\n", stdDev)
|
||||
}
|
||||
ps := h.Percentiles([]float64{0.5, 0.75, 0.99})
|
||||
if 5000.5 != ps[0] {
|
||||
t.Errorf("median: 5000.5 != %v\n", ps[0])
|
||||
}
|
||||
if 7500.75 != ps[1] {
|
||||
t.Errorf("75th percentile: 7500.75 != %v\n", ps[1])
|
||||
}
|
||||
if 9900.99 != ps[2] {
|
||||
t.Errorf("99th percentile: 9900.99 != %v\n", ps[2])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
ini "gopkg.in/ini.v1"
|
||||
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics/graphitebridge"
|
||||
)
|
||||
|
||||
var metricsLogger log.Logger = log.New("metrics")
|
||||
|
||||
type logWrapper struct {
|
||||
logger log.Logger
|
||||
}
|
||||
|
||||
func (lw *logWrapper) Println(v ...interface{}) {
|
||||
lw.logger.Info("graphite metric bridge", v...)
|
||||
}
|
||||
|
||||
func Init(file *ini.File) {
|
||||
cfg := ReadSettings(file)
|
||||
internalInit(cfg)
|
||||
}
|
||||
|
||||
func internalInit(settings *MetricSettings) {
|
||||
initMetricVars(settings)
|
||||
|
||||
if settings.GraphiteBridgeConfig != nil {
|
||||
bridge, err := graphitebridge.NewBridge(settings.GraphiteBridgeConfig)
|
||||
if err != nil {
|
||||
metricsLogger.Error("failed to create graphite bridge", "error", err)
|
||||
} else {
|
||||
go bridge.Run(context.Background())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,221 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Meters count events to produce exponentially-weighted moving average rates
|
||||
// at one-, five-, and fifteen-minutes and a mean rate.
|
||||
type Meter interface {
|
||||
Metric
|
||||
|
||||
Count() int64
|
||||
Mark(int64)
|
||||
Rate1() float64
|
||||
Rate5() float64
|
||||
Rate15() float64
|
||||
RateMean() float64
|
||||
}
|
||||
|
||||
// NewMeter constructs a new StandardMeter and launches a goroutine.
|
||||
func NewMeter(meta *MetricMeta) Meter {
|
||||
if UseNilMetrics {
|
||||
return NilMeter{}
|
||||
}
|
||||
|
||||
m := newStandardMeter(meta)
|
||||
arbiter.Lock()
|
||||
defer arbiter.Unlock()
|
||||
arbiter.meters = append(arbiter.meters, m)
|
||||
if !arbiter.started {
|
||||
arbiter.started = true
|
||||
go arbiter.tick()
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
type MeterSnapshot struct {
|
||||
*MetricMeta
|
||||
count int64
|
||||
rate1, rate5, rate15, rateMean float64
|
||||
}
|
||||
|
||||
// Count returns the count of events at the time the snapshot was taken.
|
||||
func (m *MeterSnapshot) Count() int64 { return m.count }
|
||||
|
||||
// Mark panics.
|
||||
func (*MeterSnapshot) Mark(n int64) {
|
||||
panic("Mark called on a MeterSnapshot")
|
||||
}
|
||||
|
||||
// Rate1 returns the one-minute moving average rate of events per second at the
|
||||
// time the snapshot was taken.
|
||||
func (m *MeterSnapshot) Rate1() float64 { return m.rate1 }
|
||||
|
||||
// Rate5 returns the five-minute moving average rate of events per second at
|
||||
// the time the snapshot was taken.
|
||||
func (m *MeterSnapshot) Rate5() float64 { return m.rate5 }
|
||||
|
||||
// Rate15 returns the fifteen-minute moving average rate of events per second
|
||||
// at the time the snapshot was taken.
|
||||
func (m *MeterSnapshot) Rate15() float64 { return m.rate15 }
|
||||
|
||||
// RateMean returns the meter's mean rate of events per second at the time the
|
||||
// snapshot was taken.
|
||||
func (m *MeterSnapshot) RateMean() float64 { return m.rateMean }
|
||||
|
||||
// Snapshot returns the snapshot.
|
||||
func (m *MeterSnapshot) Snapshot() Metric { return m }
|
||||
|
||||
// NilMeter is a no-op Meter.
|
||||
type NilMeter struct{ *MetricMeta }
|
||||
|
||||
// Count is a no-op.
|
||||
func (NilMeter) Count() int64 { return 0 }
|
||||
|
||||
// Mark is a no-op.
|
||||
func (NilMeter) Mark(n int64) {}
|
||||
|
||||
// Rate1 is a no-op.
|
||||
func (NilMeter) Rate1() float64 { return 0.0 }
|
||||
|
||||
// Rate5 is a no-op.
|
||||
func (NilMeter) Rate5() float64 { return 0.0 }
|
||||
|
||||
// Rate15is a no-op.
|
||||
func (NilMeter) Rate15() float64 { return 0.0 }
|
||||
|
||||
// RateMean is a no-op.
|
||||
func (NilMeter) RateMean() float64 { return 0.0 }
|
||||
|
||||
// Snapshot is a no-op.
|
||||
func (NilMeter) Snapshot() Metric { return NilMeter{} }
|
||||
|
||||
// StandardMeter is the standard implementation of a Meter.
|
||||
type StandardMeter struct {
|
||||
*MetricMeta
|
||||
lock sync.RWMutex
|
||||
snapshot *MeterSnapshot
|
||||
a1, a5, a15 EWMA
|
||||
startTime time.Time
|
||||
}
|
||||
|
||||
func newStandardMeter(meta *MetricMeta) *StandardMeter {
|
||||
return &StandardMeter{
|
||||
MetricMeta: meta,
|
||||
snapshot: &MeterSnapshot{MetricMeta: meta},
|
||||
a1: NewEWMA1(),
|
||||
a5: NewEWMA5(),
|
||||
a15: NewEWMA15(),
|
||||
startTime: time.Now(),
|
||||
}
|
||||
}
|
||||
|
||||
// Count returns the number of events recorded.
|
||||
func (m *StandardMeter) Count() int64 {
|
||||
m.lock.RLock()
|
||||
count := m.snapshot.count
|
||||
m.lock.RUnlock()
|
||||
return count
|
||||
}
|
||||
|
||||
// Mark records the occurrence of n events.
|
||||
func (m *StandardMeter) Mark(n int64) {
|
||||
m.lock.Lock()
|
||||
defer m.lock.Unlock()
|
||||
m.snapshot.count += n
|
||||
m.a1.Update(n)
|
||||
m.a5.Update(n)
|
||||
m.a15.Update(n)
|
||||
m.updateSnapshot()
|
||||
}
|
||||
|
||||
// Rate1 returns the one-minute moving average rate of events per second.
|
||||
func (m *StandardMeter) Rate1() float64 {
|
||||
m.lock.RLock()
|
||||
rate1 := m.snapshot.rate1
|
||||
m.lock.RUnlock()
|
||||
return rate1
|
||||
}
|
||||
|
||||
// Rate5 returns the five-minute moving average rate of events per second.
|
||||
func (m *StandardMeter) Rate5() float64 {
|
||||
m.lock.RLock()
|
||||
rate5 := m.snapshot.rate5
|
||||
m.lock.RUnlock()
|
||||
return rate5
|
||||
}
|
||||
|
||||
// Rate15 returns the fifteen-minute moving average rate of events per second.
|
||||
func (m *StandardMeter) Rate15() float64 {
|
||||
m.lock.RLock()
|
||||
rate15 := m.snapshot.rate15
|
||||
m.lock.RUnlock()
|
||||
return rate15
|
||||
}
|
||||
|
||||
// RateMean returns the meter's mean rate of events per second.
|
||||
func (m *StandardMeter) RateMean() float64 {
|
||||
m.lock.RLock()
|
||||
rateMean := m.snapshot.rateMean
|
||||
m.lock.RUnlock()
|
||||
return rateMean
|
||||
}
|
||||
|
||||
// Snapshot returns a read-only copy of the meter.
|
||||
func (m *StandardMeter) Snapshot() Metric {
|
||||
m.lock.RLock()
|
||||
snapshot := *m.snapshot
|
||||
m.lock.RUnlock()
|
||||
return &snapshot
|
||||
}
|
||||
|
||||
func (m *StandardMeter) updateSnapshot() {
|
||||
// should run with write lock held on m.lock
|
||||
snapshot := m.snapshot
|
||||
snapshot.rate1 = m.a1.Rate()
|
||||
snapshot.rate5 = m.a5.Rate()
|
||||
snapshot.rate15 = m.a15.Rate()
|
||||
snapshot.rateMean = float64(snapshot.count) / time.Since(m.startTime).Seconds()
|
||||
}
|
||||
|
||||
func (m *StandardMeter) tick() {
|
||||
m.lock.Lock()
|
||||
defer m.lock.Unlock()
|
||||
m.a1.Tick()
|
||||
m.a5.Tick()
|
||||
m.a15.Tick()
|
||||
m.updateSnapshot()
|
||||
}
|
||||
|
||||
type meterArbiter struct {
|
||||
sync.RWMutex
|
||||
started bool
|
||||
meters []*StandardMeter
|
||||
ticker *time.Ticker
|
||||
}
|
||||
|
||||
var arbiter = meterArbiter{ticker: time.NewTicker(5e9)}
|
||||
|
||||
// Ticks meters on the scheduled interval
|
||||
func (ma *meterArbiter) tick() {
|
||||
for {
|
||||
select {
|
||||
case <-ma.ticker.C:
|
||||
ma.tickMeters()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (ma *meterArbiter) tickMeters() {
|
||||
ma.RLock()
|
||||
defer ma.RUnlock()
|
||||
for _, meter := range ma.meters {
|
||||
meter.tick()
|
||||
}
|
||||
}
|
||||
+385
-133
@@ -1,146 +1,398 @@
|
||||
package metrics
|
||||
|
||||
var MetricStats Registry
|
||||
var UseNilMetrics bool
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
func init() {
|
||||
// init with nil metrics
|
||||
initMetricVars(&MetricSettings{})
|
||||
}
|
||||
|
||||
var (
|
||||
M_Instance_Start Counter
|
||||
M_Page_Status_200 Counter
|
||||
M_Page_Status_500 Counter
|
||||
M_Page_Status_404 Counter
|
||||
M_Page_Status_Unknown Counter
|
||||
M_Api_Status_200 Counter
|
||||
M_Api_Status_404 Counter
|
||||
M_Api_Status_500 Counter
|
||||
M_Api_Status_Unknown Counter
|
||||
M_Proxy_Status_200 Counter
|
||||
M_Proxy_Status_404 Counter
|
||||
M_Proxy_Status_500 Counter
|
||||
M_Proxy_Status_Unknown Counter
|
||||
M_Api_User_SignUpStarted Counter
|
||||
M_Api_User_SignUpCompleted Counter
|
||||
M_Api_User_SignUpInvite Counter
|
||||
M_Api_Dashboard_Save Timer
|
||||
M_Api_Dashboard_Get Timer
|
||||
M_Api_Dashboard_Search Timer
|
||||
M_Api_Admin_User_Create Counter
|
||||
M_Api_Login_Post Counter
|
||||
M_Api_Login_OAuth Counter
|
||||
M_Api_Org_Create Counter
|
||||
M_Api_Dashboard_Snapshot_Create Counter
|
||||
M_Api_Dashboard_Snapshot_External Counter
|
||||
M_Api_Dashboard_Snapshot_Get Counter
|
||||
M_Models_Dashboard_Insert Counter
|
||||
M_Alerting_Result_State_Alerting Counter
|
||||
M_Alerting_Result_State_Ok Counter
|
||||
M_Alerting_Result_State_Paused Counter
|
||||
M_Alerting_Result_State_NoData Counter
|
||||
M_Alerting_Result_State_Pending Counter
|
||||
M_Alerting_Notification_Sent_Slack Counter
|
||||
M_Alerting_Notification_Sent_Email Counter
|
||||
M_Alerting_Notification_Sent_Webhook Counter
|
||||
M_Alerting_Notification_Sent_DingDing Counter
|
||||
M_Alerting_Notification_Sent_PagerDuty Counter
|
||||
M_Alerting_Notification_Sent_LINE Counter
|
||||
M_Alerting_Notification_Sent_Victorops Counter
|
||||
M_Alerting_Notification_Sent_OpsGenie Counter
|
||||
M_Alerting_Notification_Sent_Telegram Counter
|
||||
M_Alerting_Notification_Sent_Threema Counter
|
||||
M_Alerting_Notification_Sent_Sensu Counter
|
||||
M_Alerting_Notification_Sent_Pushover Counter
|
||||
M_Aws_CloudWatch_GetMetricStatistics Counter
|
||||
M_Aws_CloudWatch_ListMetrics Counter
|
||||
M_DB_DataSource_QueryById Counter
|
||||
|
||||
// Timers
|
||||
M_DataSource_ProxyReq_Timer Timer
|
||||
M_Alerting_Execution_Time Timer
|
||||
|
||||
// StatTotals
|
||||
M_Alerting_Active_Alerts Gauge
|
||||
M_StatTotal_Dashboards Gauge
|
||||
M_StatTotal_Users Gauge
|
||||
M_StatTotal_Orgs Gauge
|
||||
M_StatTotal_Playlists Gauge
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/plugins"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
)
|
||||
|
||||
func initMetricVars(settings *MetricSettings) {
|
||||
UseNilMetrics = settings.Enabled == false
|
||||
MetricStats = NewRegistry()
|
||||
const exporterName = "grafana"
|
||||
|
||||
M_Instance_Start = RegCounter("instance_start")
|
||||
var (
|
||||
M_Instance_Start prometheus.Counter
|
||||
M_Page_Status *prometheus.CounterVec
|
||||
M_Api_Status *prometheus.CounterVec
|
||||
M_Proxy_Status *prometheus.CounterVec
|
||||
M_Http_Request_Total *prometheus.CounterVec
|
||||
M_Http_Request_Summary *prometheus.SummaryVec
|
||||
|
||||
M_Page_Status_200 = RegCounter("page.resp_status", "code", "200")
|
||||
M_Page_Status_500 = RegCounter("page.resp_status", "code", "500")
|
||||
M_Page_Status_404 = RegCounter("page.resp_status", "code", "404")
|
||||
M_Page_Status_Unknown = RegCounter("page.resp_status", "code", "unknown")
|
||||
M_Api_User_SignUpStarted prometheus.Counter
|
||||
M_Api_User_SignUpCompleted prometheus.Counter
|
||||
M_Api_User_SignUpInvite prometheus.Counter
|
||||
M_Api_Dashboard_Save prometheus.Summary
|
||||
M_Api_Dashboard_Get prometheus.Summary
|
||||
M_Api_Dashboard_Search prometheus.Summary
|
||||
M_Api_Admin_User_Create prometheus.Counter
|
||||
M_Api_Login_Post prometheus.Counter
|
||||
M_Api_Login_OAuth prometheus.Counter
|
||||
M_Api_Org_Create prometheus.Counter
|
||||
|
||||
M_Api_Status_200 = RegCounter("api.resp_status", "code", "200")
|
||||
M_Api_Status_404 = RegCounter("api.resp_status", "code", "404")
|
||||
M_Api_Status_500 = RegCounter("api.resp_status", "code", "500")
|
||||
M_Api_Status_Unknown = RegCounter("api.resp_status", "code", "unknown")
|
||||
|
||||
M_Proxy_Status_200 = RegCounter("proxy.resp_status", "code", "200")
|
||||
M_Proxy_Status_404 = RegCounter("proxy.resp_status", "code", "404")
|
||||
M_Proxy_Status_500 = RegCounter("proxy.resp_status", "code", "500")
|
||||
M_Proxy_Status_Unknown = RegCounter("proxy.resp_status", "code", "unknown")
|
||||
|
||||
M_Api_User_SignUpStarted = RegCounter("api.user.signup_started")
|
||||
M_Api_User_SignUpCompleted = RegCounter("api.user.signup_completed")
|
||||
M_Api_User_SignUpInvite = RegCounter("api.user.signup_invite")
|
||||
|
||||
M_Api_Dashboard_Save = RegTimer("api.dashboard.save")
|
||||
M_Api_Dashboard_Get = RegTimer("api.dashboard.get")
|
||||
M_Api_Dashboard_Search = RegTimer("api.dashboard.search")
|
||||
|
||||
M_Api_Admin_User_Create = RegCounter("api.admin.user_create")
|
||||
M_Api_Login_Post = RegCounter("api.login.post")
|
||||
M_Api_Login_OAuth = RegCounter("api.login.oauth")
|
||||
M_Api_Org_Create = RegCounter("api.org.create")
|
||||
|
||||
M_Api_Dashboard_Snapshot_Create = RegCounter("api.dashboard_snapshot.create")
|
||||
M_Api_Dashboard_Snapshot_External = RegCounter("api.dashboard_snapshot.external")
|
||||
M_Api_Dashboard_Snapshot_Get = RegCounter("api.dashboard_snapshot.get")
|
||||
|
||||
M_Models_Dashboard_Insert = RegCounter("models.dashboard.insert")
|
||||
|
||||
M_Alerting_Result_State_Alerting = RegCounter("alerting.result", "state", "alerting")
|
||||
M_Alerting_Result_State_Ok = RegCounter("alerting.result", "state", "ok")
|
||||
M_Alerting_Result_State_Paused = RegCounter("alerting.result", "state", "paused")
|
||||
M_Alerting_Result_State_NoData = RegCounter("alerting.result", "state", "no_data")
|
||||
M_Alerting_Result_State_Pending = RegCounter("alerting.result", "state", "pending")
|
||||
|
||||
M_Alerting_Notification_Sent_Slack = RegCounter("alerting.notifications_sent", "type", "slack")
|
||||
M_Alerting_Notification_Sent_Email = RegCounter("alerting.notifications_sent", "type", "email")
|
||||
M_Alerting_Notification_Sent_Webhook = RegCounter("alerting.notifications_sent", "type", "webhook")
|
||||
M_Alerting_Notification_Sent_DingDing = RegCounter("alerting.notifications_sent", "type", "dingding")
|
||||
M_Alerting_Notification_Sent_PagerDuty = RegCounter("alerting.notifications_sent", "type", "pagerduty")
|
||||
M_Alerting_Notification_Sent_Victorops = RegCounter("alerting.notifications_sent", "type", "victorops")
|
||||
M_Alerting_Notification_Sent_OpsGenie = RegCounter("alerting.notifications_sent", "type", "opsgenie")
|
||||
M_Alerting_Notification_Sent_Telegram = RegCounter("alerting.notifications_sent", "type", "telegram")
|
||||
M_Alerting_Notification_Sent_Threema = RegCounter("alerting.notifications_sent", "type", "threema")
|
||||
M_Alerting_Notification_Sent_Sensu = RegCounter("alerting.notifications_sent", "type", "sensu")
|
||||
M_Alerting_Notification_Sent_LINE = RegCounter("alerting.notifications_sent", "type", "LINE")
|
||||
M_Alerting_Notification_Sent_Pushover = RegCounter("alerting.notifications_sent", "type", "pushover")
|
||||
|
||||
M_Aws_CloudWatch_GetMetricStatistics = RegCounter("aws.cloudwatch.get_metric_statistics")
|
||||
M_Aws_CloudWatch_ListMetrics = RegCounter("aws.cloudwatch.list_metrics")
|
||||
|
||||
M_DB_DataSource_QueryById = RegCounter("db.datasource.query_by_id")
|
||||
M_Api_Dashboard_Snapshot_Create prometheus.Counter
|
||||
M_Api_Dashboard_Snapshot_External prometheus.Counter
|
||||
M_Api_Dashboard_Snapshot_Get prometheus.Counter
|
||||
M_Api_Dashboard_Insert prometheus.Counter
|
||||
M_Alerting_Result_State *prometheus.CounterVec
|
||||
M_Alerting_Notification_Sent *prometheus.CounterVec
|
||||
M_Aws_CloudWatch_GetMetricStatistics prometheus.Counter
|
||||
M_Aws_CloudWatch_ListMetrics prometheus.Counter
|
||||
M_DB_DataSource_QueryById prometheus.Counter
|
||||
|
||||
// Timers
|
||||
M_DataSource_ProxyReq_Timer = RegTimer("api.dataproxy.request.all")
|
||||
M_Alerting_Execution_Time = RegTimer("alerting.execution_time")
|
||||
M_DataSource_ProxyReq_Timer prometheus.Summary
|
||||
M_Alerting_Execution_Time prometheus.Summary
|
||||
|
||||
// StatTotals
|
||||
M_Alerting_Active_Alerts = RegGauge("alerting.active_alerts")
|
||||
M_StatTotal_Dashboards = RegGauge("stat_totals", "stat", "dashboards")
|
||||
M_StatTotal_Users = RegGauge("stat_totals", "stat", "users")
|
||||
M_StatTotal_Orgs = RegGauge("stat_totals", "stat", "orgs")
|
||||
M_StatTotal_Playlists = RegGauge("stat_totals", "stat", "playlists")
|
||||
M_Alerting_Active_Alerts prometheus.Gauge
|
||||
M_StatTotal_Dashboards prometheus.Gauge
|
||||
M_StatTotal_Users prometheus.Gauge
|
||||
M_StatTotal_Orgs prometheus.Gauge
|
||||
M_StatTotal_Playlists prometheus.Gauge
|
||||
)
|
||||
|
||||
func init() {
|
||||
M_Instance_Start = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "instance_start_total",
|
||||
Help: "counter for started instances",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Page_Status = prometheus.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Name: "page_response_status_total",
|
||||
Help: "page http response status",
|
||||
Namespace: exporterName,
|
||||
},
|
||||
[]string{"code"},
|
||||
)
|
||||
|
||||
M_Api_Status = prometheus.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Name: "api_response_status_total",
|
||||
Help: "api http response status",
|
||||
Namespace: exporterName,
|
||||
},
|
||||
[]string{"code"},
|
||||
)
|
||||
|
||||
M_Proxy_Status = prometheus.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Name: "proxy_response_status_total",
|
||||
Help: "proxy http response status",
|
||||
Namespace: exporterName,
|
||||
},
|
||||
[]string{"code"},
|
||||
)
|
||||
|
||||
M_Http_Request_Total = prometheus.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Name: "http_request_total",
|
||||
Help: "http request counter",
|
||||
},
|
||||
[]string{"handler", "statuscode", "method"},
|
||||
)
|
||||
|
||||
M_Http_Request_Summary = prometheus.NewSummaryVec(
|
||||
prometheus.SummaryOpts{
|
||||
Name: "http_request_duration",
|
||||
Help: "http request summary",
|
||||
},
|
||||
[]string{"handler", "statuscode", "method"},
|
||||
)
|
||||
|
||||
M_Api_User_SignUpStarted = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_user_signup_started_total",
|
||||
Help: "amount of users who started the signup flow",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_User_SignUpCompleted = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_user_signup_completed_total",
|
||||
Help: "amount of users who completed the signup flow",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_User_SignUpInvite = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_user_signup_invite_total",
|
||||
Help: "amount of users who have been invited",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Save = prometheus.NewSummary(prometheus.SummaryOpts{
|
||||
Name: "api_dashboard_save",
|
||||
Help: "summary for dashboard save duration",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Get = prometheus.NewSummary(prometheus.SummaryOpts{
|
||||
Name: "api_dashboard_get",
|
||||
Help: "summary for dashboard get duration",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Search = prometheus.NewSummary(prometheus.SummaryOpts{
|
||||
Name: "api_dashboard_search",
|
||||
Help: "summary for dashboard search duration",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Admin_User_Create = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_admin_user_created_total",
|
||||
Help: "api admin user created counter",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Login_Post = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_login_post_total",
|
||||
Help: "api login post counter",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Login_OAuth = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_login_oauth_total",
|
||||
Help: "api login oauth counter",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Org_Create = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_org_create_total",
|
||||
Help: "api org created counter",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Snapshot_Create = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_dashboard_snapshot_create_total",
|
||||
Help: "dashboard snapshots created",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Snapshot_External = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_dashboard_snapshot_external_total",
|
||||
Help: "external dashboard snapshots created",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Snapshot_Get = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_dashboard_snapshot_get_total",
|
||||
Help: "loaded dashboards",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Api_Dashboard_Insert = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "api_models_dashboard_insert_total",
|
||||
Help: "dashboards inserted ",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Alerting_Result_State = prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||
Name: "alerting_result_total",
|
||||
Help: "alert execution result counter",
|
||||
Namespace: exporterName,
|
||||
}, []string{"state"})
|
||||
|
||||
M_Alerting_Notification_Sent = prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||
Name: "alerting_notification_sent_total",
|
||||
Help: "counter for how many alert notifications been sent",
|
||||
Namespace: exporterName,
|
||||
}, []string{"type"})
|
||||
|
||||
M_Aws_CloudWatch_GetMetricStatistics = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "aws_cloudwatch_get_metric_statistics_total",
|
||||
Help: "counter for getting metric statistics from aws",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Aws_CloudWatch_ListMetrics = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "aws_cloudwatch_list_metrics_total",
|
||||
Help: "counter for getting list of metrics from aws",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_DB_DataSource_QueryById = prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "db_datasource_query_by_id_total",
|
||||
Help: "counter for getting datasource by id",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_DataSource_ProxyReq_Timer = prometheus.NewSummary(prometheus.SummaryOpts{
|
||||
Name: "api_dataproxy_request_all",
|
||||
Help: "summary for dashboard search duration",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Alerting_Execution_Time = prometheus.NewSummary(prometheus.SummaryOpts{
|
||||
Name: "alerting_execution_time_seconds",
|
||||
Help: "summary of alert exeuction duration",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_Alerting_Active_Alerts = prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "alerting_active_alerts",
|
||||
Help: "amount of active alerts",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_StatTotal_Dashboards = prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "stat_totals_dashboard",
|
||||
Help: "total amount of dashboards",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_StatTotal_Users = prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "stat_total_users",
|
||||
Help: "total amount of users",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_StatTotal_Orgs = prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "stat_total_orgs",
|
||||
Help: "total amount of orgs",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
|
||||
M_StatTotal_Playlists = prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "stat_total_playlists",
|
||||
Help: "total amount of playlists",
|
||||
Namespace: exporterName,
|
||||
})
|
||||
}
|
||||
|
||||
func initMetricVars(settings *MetricSettings) {
|
||||
prometheus.MustRegister(
|
||||
M_Instance_Start,
|
||||
M_Page_Status,
|
||||
M_Api_Status,
|
||||
M_Proxy_Status,
|
||||
M_Http_Request_Total,
|
||||
M_Http_Request_Summary,
|
||||
M_Api_User_SignUpStarted,
|
||||
M_Api_User_SignUpCompleted,
|
||||
M_Api_User_SignUpInvite,
|
||||
M_Api_Dashboard_Save,
|
||||
M_Api_Dashboard_Get,
|
||||
M_Api_Dashboard_Search,
|
||||
M_DataSource_ProxyReq_Timer,
|
||||
M_Alerting_Execution_Time,
|
||||
M_Api_Admin_User_Create,
|
||||
M_Api_Login_Post,
|
||||
M_Api_Login_OAuth,
|
||||
M_Api_Org_Create,
|
||||
M_Api_Dashboard_Snapshot_Create,
|
||||
M_Api_Dashboard_Snapshot_External,
|
||||
M_Api_Dashboard_Snapshot_Get,
|
||||
M_Api_Dashboard_Insert,
|
||||
M_Alerting_Result_State,
|
||||
M_Alerting_Notification_Sent,
|
||||
M_Aws_CloudWatch_GetMetricStatistics,
|
||||
M_Aws_CloudWatch_ListMetrics,
|
||||
M_DB_DataSource_QueryById,
|
||||
M_Alerting_Active_Alerts,
|
||||
M_StatTotal_Dashboards,
|
||||
M_StatTotal_Users,
|
||||
M_StatTotal_Orgs,
|
||||
M_StatTotal_Playlists)
|
||||
|
||||
go instrumentationLoop(settings)
|
||||
}
|
||||
|
||||
func instrumentationLoop(settings *MetricSettings) chan struct{} {
|
||||
M_Instance_Start.Inc()
|
||||
|
||||
onceEveryDayTick := time.NewTicker(time.Hour * 24)
|
||||
secondTicker := time.NewTicker(time.Second * time.Duration(settings.IntervalSeconds))
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-onceEveryDayTick.C:
|
||||
sendUsageStats()
|
||||
case <-secondTicker.C:
|
||||
updateTotalStats()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var metricPublishCounter int64 = 0
|
||||
|
||||
func updateTotalStats() {
|
||||
metricPublishCounter++
|
||||
if metricPublishCounter == 1 || metricPublishCounter%10 == 0 {
|
||||
statsQuery := models.GetSystemStatsQuery{}
|
||||
if err := bus.Dispatch(&statsQuery); err != nil {
|
||||
metricsLogger.Error("Failed to get system stats", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
M_StatTotal_Dashboards.Set(float64(statsQuery.Result.Dashboards))
|
||||
M_StatTotal_Users.Set(float64(statsQuery.Result.Users))
|
||||
M_StatTotal_Playlists.Set(float64(statsQuery.Result.Playlists))
|
||||
M_StatTotal_Orgs.Set(float64(statsQuery.Result.Orgs))
|
||||
}
|
||||
}
|
||||
|
||||
func sendUsageStats() {
|
||||
if !setting.ReportingEnabled {
|
||||
return
|
||||
}
|
||||
|
||||
metricsLogger.Debug("Sending anonymous usage stats to stats.grafana.org")
|
||||
|
||||
version := strings.Replace(setting.BuildVersion, ".", "_", -1)
|
||||
|
||||
metrics := map[string]interface{}{}
|
||||
report := map[string]interface{}{
|
||||
"version": version,
|
||||
"metrics": metrics,
|
||||
"os": runtime.GOOS,
|
||||
"arch": runtime.GOARCH,
|
||||
}
|
||||
|
||||
statsQuery := models.GetSystemStatsQuery{}
|
||||
if err := bus.Dispatch(&statsQuery); err != nil {
|
||||
metricsLogger.Error("Failed to get system stats", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
metrics["stats.dashboards.count"] = statsQuery.Result.Dashboards
|
||||
metrics["stats.users.count"] = statsQuery.Result.Users
|
||||
metrics["stats.orgs.count"] = statsQuery.Result.Orgs
|
||||
metrics["stats.playlist.count"] = statsQuery.Result.Playlists
|
||||
metrics["stats.plugins.apps.count"] = len(plugins.Apps)
|
||||
metrics["stats.plugins.panels.count"] = len(plugins.Panels)
|
||||
metrics["stats.plugins.datasources.count"] = len(plugins.DataSources)
|
||||
metrics["stats.alerts.count"] = statsQuery.Result.Alerts
|
||||
metrics["stats.active_users.count"] = statsQuery.Result.ActiveUsers
|
||||
metrics["stats.datasources.count"] = statsQuery.Result.Datasources
|
||||
|
||||
dsStats := models.GetDataSourceStatsQuery{}
|
||||
if err := bus.Dispatch(&dsStats); err != nil {
|
||||
metricsLogger.Error("Failed to get datasource stats", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
// send counters for each data source
|
||||
// but ignore any custom data sources
|
||||
// as sending that name could be sensitive information
|
||||
dsOtherCount := 0
|
||||
for _, dsStat := range dsStats.Result {
|
||||
if models.IsKnownDataSourcePlugin(dsStat.Type) {
|
||||
metrics["stats.ds."+dsStat.Type+".count"] = dsStat.Count
|
||||
} else {
|
||||
dsOtherCount += dsStat.Count
|
||||
}
|
||||
}
|
||||
metrics["stats.ds.other.count"] = dsOtherCount
|
||||
|
||||
out, _ := json.MarshalIndent(report, "", " ")
|
||||
data := bytes.NewBuffer(out)
|
||||
|
||||
client := http.Client{Timeout: time.Duration(5 * time.Second)}
|
||||
go client.Post("https://stats.grafana.org/grafana-usage-report", "application/json", data)
|
||||
}
|
||||
|
||||
@@ -1,135 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/plugins"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
)
|
||||
|
||||
var metricsLogger log.Logger = log.New("metrics")
|
||||
var metricPublishCounter int64 = 0
|
||||
|
||||
func Init() {
|
||||
settings := readSettings()
|
||||
initMetricVars(settings)
|
||||
go instrumentationLoop(settings)
|
||||
}
|
||||
|
||||
func instrumentationLoop(settings *MetricSettings) chan struct{} {
|
||||
M_Instance_Start.Inc(1)
|
||||
|
||||
onceEveryDayTick := time.NewTicker(time.Hour * 24)
|
||||
secondTicker := time.NewTicker(time.Second * time.Duration(settings.IntervalSeconds))
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-onceEveryDayTick.C:
|
||||
sendUsageStats()
|
||||
case <-secondTicker.C:
|
||||
if settings.Enabled {
|
||||
sendMetrics(settings)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func sendMetrics(settings *MetricSettings) {
|
||||
if len(settings.Publishers) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
updateTotalStats()
|
||||
|
||||
metrics := MetricStats.GetSnapshots()
|
||||
for _, publisher := range settings.Publishers {
|
||||
publisher.Publish(metrics)
|
||||
}
|
||||
}
|
||||
|
||||
func updateTotalStats() {
|
||||
|
||||
// every interval also publish totals
|
||||
metricPublishCounter++
|
||||
if metricPublishCounter%10 == 0 {
|
||||
// get stats
|
||||
statsQuery := m.GetSystemStatsQuery{}
|
||||
if err := bus.Dispatch(&statsQuery); err != nil {
|
||||
metricsLogger.Error("Failed to get system stats", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
M_StatTotal_Dashboards.Update(statsQuery.Result.Dashboards)
|
||||
M_StatTotal_Users.Update(statsQuery.Result.Users)
|
||||
M_StatTotal_Playlists.Update(statsQuery.Result.Playlists)
|
||||
M_StatTotal_Orgs.Update(statsQuery.Result.Orgs)
|
||||
}
|
||||
}
|
||||
|
||||
func sendUsageStats() {
|
||||
if !setting.ReportingEnabled {
|
||||
return
|
||||
}
|
||||
|
||||
metricsLogger.Debug("Sending anonymous usage stats to stats.grafana.org")
|
||||
|
||||
version := strings.Replace(setting.BuildVersion, ".", "_", -1)
|
||||
|
||||
metrics := map[string]interface{}{}
|
||||
report := map[string]interface{}{
|
||||
"version": version,
|
||||
"metrics": metrics,
|
||||
"os": runtime.GOOS,
|
||||
"arch": runtime.GOARCH,
|
||||
}
|
||||
|
||||
statsQuery := m.GetSystemStatsQuery{}
|
||||
if err := bus.Dispatch(&statsQuery); err != nil {
|
||||
metricsLogger.Error("Failed to get system stats", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
metrics["stats.dashboards.count"] = statsQuery.Result.Dashboards
|
||||
metrics["stats.users.count"] = statsQuery.Result.Users
|
||||
metrics["stats.orgs.count"] = statsQuery.Result.Orgs
|
||||
metrics["stats.playlist.count"] = statsQuery.Result.Playlists
|
||||
metrics["stats.plugins.apps.count"] = len(plugins.Apps)
|
||||
metrics["stats.plugins.panels.count"] = len(plugins.Panels)
|
||||
metrics["stats.plugins.datasources.count"] = len(plugins.DataSources)
|
||||
metrics["stats.alerts.count"] = statsQuery.Result.Alerts
|
||||
metrics["stats.active_users.count"] = statsQuery.Result.ActiveUsers
|
||||
metrics["stats.datasources.count"] = statsQuery.Result.Datasources
|
||||
|
||||
dsStats := m.GetDataSourceStatsQuery{}
|
||||
if err := bus.Dispatch(&dsStats); err != nil {
|
||||
metricsLogger.Error("Failed to get datasource stats", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
// send counters for each data source
|
||||
// but ignore any custom data sources
|
||||
// as sending that name could be sensitive information
|
||||
dsOtherCount := 0
|
||||
for _, dsStat := range dsStats.Result {
|
||||
if m.IsKnownDataSourcePlugin(dsStat.Type) {
|
||||
metrics["stats.ds."+dsStat.Type+".count"] = dsStat.Count
|
||||
} else {
|
||||
dsOtherCount += dsStat.Count
|
||||
}
|
||||
}
|
||||
metrics["stats.ds.other.count"] = dsOtherCount
|
||||
|
||||
out, _ := json.MarshalIndent(report, "", " ")
|
||||
data := bytes.NewBuffer(out)
|
||||
|
||||
client := http.Client{Timeout: time.Duration(5 * time.Second)}
|
||||
go client.Post("https://stats.grafana.org/grafana-usage-report", "application/json", data)
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
package metrics
|
||||
|
||||
import "sync"
|
||||
|
||||
type Registry interface {
|
||||
GetSnapshots() []Metric
|
||||
Register(metric Metric)
|
||||
}
|
||||
|
||||
// The standard implementation of a Registry is a mutex-protected map
|
||||
// of names to metrics.
|
||||
type StandardRegistry struct {
|
||||
metrics []Metric
|
||||
mutex sync.Mutex
|
||||
}
|
||||
|
||||
// Create a new registry.
|
||||
func NewRegistry() Registry {
|
||||
return &StandardRegistry{
|
||||
metrics: make([]Metric, 0),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *StandardRegistry) Register(metric Metric) {
|
||||
r.mutex.Lock()
|
||||
defer r.mutex.Unlock()
|
||||
r.metrics = append(r.metrics, metric)
|
||||
}
|
||||
|
||||
// Call the given function for each registered metric.
|
||||
func (r *StandardRegistry) GetSnapshots() []Metric {
|
||||
metrics := make([]Metric, len(r.metrics))
|
||||
for i, metric := range r.metrics {
|
||||
metrics[i] = metric.Snapshot()
|
||||
}
|
||||
return metrics
|
||||
}
|
||||
@@ -1,607 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"math"
|
||||
"math/rand"
|
||||
"sort"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const rescaleThreshold = time.Hour
|
||||
|
||||
// Samples maintain a statistically-significant selection of values from
|
||||
// a stream.
|
||||
type Sample interface {
|
||||
Clear()
|
||||
Count() int64
|
||||
Max() int64
|
||||
Mean() float64
|
||||
Min() int64
|
||||
Percentile(float64) float64
|
||||
Percentiles([]float64) []float64
|
||||
Size() int
|
||||
Snapshot() Sample
|
||||
StdDev() float64
|
||||
Sum() int64
|
||||
Update(int64)
|
||||
Values() []int64
|
||||
Variance() float64
|
||||
}
|
||||
|
||||
// ExpDecaySample is an exponentially-decaying sample using a forward-decaying
|
||||
// priority reservoir. See Cormode et al's "Forward Decay: A Practical Time
|
||||
// Decay Model for Streaming Systems".
|
||||
//
|
||||
// <http://www.research.att.com/people/Cormode_Graham/library/publications/CormodeShkapenyukSrivastavaXu09.pdf>
|
||||
type ExpDecaySample struct {
|
||||
alpha float64
|
||||
count int64
|
||||
mutex sync.Mutex
|
||||
reservoirSize int
|
||||
t0, t1 time.Time
|
||||
values *expDecaySampleHeap
|
||||
}
|
||||
|
||||
// NewExpDecaySample constructs a new exponentially-decaying sample with the
|
||||
// given reservoir size and alpha.
|
||||
func NewExpDecaySample(reservoirSize int, alpha float64) Sample {
|
||||
s := &ExpDecaySample{
|
||||
alpha: alpha,
|
||||
reservoirSize: reservoirSize,
|
||||
t0: time.Now(),
|
||||
values: newExpDecaySampleHeap(reservoirSize),
|
||||
}
|
||||
s.t1 = s.t0.Add(rescaleThreshold)
|
||||
return s
|
||||
}
|
||||
|
||||
// Clear clears all samples.
|
||||
func (s *ExpDecaySample) Clear() {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
s.count = 0
|
||||
s.t0 = time.Now()
|
||||
s.t1 = s.t0.Add(rescaleThreshold)
|
||||
s.values.Clear()
|
||||
}
|
||||
|
||||
// Count returns the number of samples recorded, which may exceed the
|
||||
// reservoir size.
|
||||
func (s *ExpDecaySample) Count() int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return s.count
|
||||
}
|
||||
|
||||
// Max returns the maximum value in the sample, which may not be the maximum
|
||||
// value ever to be part of the sample.
|
||||
func (s *ExpDecaySample) Max() int64 {
|
||||
return SampleMax(s.Values())
|
||||
}
|
||||
|
||||
// Mean returns the mean of the values in the sample.
|
||||
func (s *ExpDecaySample) Mean() float64 {
|
||||
return SampleMean(s.Values())
|
||||
}
|
||||
|
||||
// Min returns the minimum value in the sample, which may not be the minimum
|
||||
// value ever to be part of the sample.
|
||||
func (s *ExpDecaySample) Min() int64 {
|
||||
return SampleMin(s.Values())
|
||||
}
|
||||
|
||||
// Percentile returns an arbitrary percentile of values in the sample.
|
||||
func (s *ExpDecaySample) Percentile(p float64) float64 {
|
||||
return SamplePercentile(s.Values(), p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of values in the
|
||||
// sample.
|
||||
func (s *ExpDecaySample) Percentiles(ps []float64) []float64 {
|
||||
return SamplePercentiles(s.Values(), ps)
|
||||
}
|
||||
|
||||
// Size returns the size of the sample, which is at most the reservoir size.
|
||||
func (s *ExpDecaySample) Size() int {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return s.values.Size()
|
||||
}
|
||||
|
||||
// Snapshot returns a read-only copy of the sample.
|
||||
func (s *ExpDecaySample) Snapshot() Sample {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
vals := s.values.Values()
|
||||
values := make([]int64, len(vals))
|
||||
for i, v := range vals {
|
||||
values[i] = v.v
|
||||
}
|
||||
return &SampleSnapshot{
|
||||
count: s.count,
|
||||
values: values,
|
||||
}
|
||||
}
|
||||
|
||||
// StdDev returns the standard deviation of the values in the sample.
|
||||
func (s *ExpDecaySample) StdDev() float64 {
|
||||
return SampleStdDev(s.Values())
|
||||
}
|
||||
|
||||
// Sum returns the sum of the values in the sample.
|
||||
func (s *ExpDecaySample) Sum() int64 {
|
||||
return SampleSum(s.Values())
|
||||
}
|
||||
|
||||
// Update samples a new value.
|
||||
func (s *ExpDecaySample) Update(v int64) {
|
||||
s.update(time.Now(), v)
|
||||
}
|
||||
|
||||
// Values returns a copy of the values in the sample.
|
||||
func (s *ExpDecaySample) Values() []int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
vals := s.values.Values()
|
||||
values := make([]int64, len(vals))
|
||||
for i, v := range vals {
|
||||
values[i] = v.v
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
// Variance returns the variance of the values in the sample.
|
||||
func (s *ExpDecaySample) Variance() float64 {
|
||||
return SampleVariance(s.Values())
|
||||
}
|
||||
|
||||
// update samples a new value at a particular timestamp. This is a method all
|
||||
// its own to facilitate testing.
|
||||
func (s *ExpDecaySample) update(t time.Time, v int64) {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
s.count++
|
||||
if s.values.Size() == s.reservoirSize {
|
||||
s.values.Pop()
|
||||
}
|
||||
s.values.Push(expDecaySample{
|
||||
k: math.Exp(t.Sub(s.t0).Seconds()*s.alpha) / rand.Float64(),
|
||||
v: v,
|
||||
})
|
||||
if t.After(s.t1) {
|
||||
values := s.values.Values()
|
||||
t0 := s.t0
|
||||
s.values.Clear()
|
||||
s.t0 = t
|
||||
s.t1 = s.t0.Add(rescaleThreshold)
|
||||
for _, v := range values {
|
||||
v.k = v.k * math.Exp(-s.alpha*s.t0.Sub(t0).Seconds())
|
||||
s.values.Push(v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// NilSample is a no-op Sample.
|
||||
type NilSample struct{}
|
||||
|
||||
// Clear is a no-op.
|
||||
func (NilSample) Clear() {}
|
||||
|
||||
// Count is a no-op.
|
||||
func (NilSample) Count() int64 { return 0 }
|
||||
|
||||
// Max is a no-op.
|
||||
func (NilSample) Max() int64 { return 0 }
|
||||
|
||||
// Mean is a no-op.
|
||||
func (NilSample) Mean() float64 { return 0.0 }
|
||||
|
||||
// Min is a no-op.
|
||||
func (NilSample) Min() int64 { return 0 }
|
||||
|
||||
// Percentile is a no-op.
|
||||
func (NilSample) Percentile(p float64) float64 { return 0.0 }
|
||||
|
||||
// Percentiles is a no-op.
|
||||
func (NilSample) Percentiles(ps []float64) []float64 {
|
||||
return make([]float64, len(ps))
|
||||
}
|
||||
|
||||
// Size is a no-op.
|
||||
func (NilSample) Size() int { return 0 }
|
||||
|
||||
// Sample is a no-op.
|
||||
func (NilSample) Snapshot() Sample { return NilSample{} }
|
||||
|
||||
// StdDev is a no-op.
|
||||
func (NilSample) StdDev() float64 { return 0.0 }
|
||||
|
||||
// Sum is a no-op.
|
||||
func (NilSample) Sum() int64 { return 0 }
|
||||
|
||||
// Update is a no-op.
|
||||
func (NilSample) Update(v int64) {}
|
||||
|
||||
// Values is a no-op.
|
||||
func (NilSample) Values() []int64 { return []int64{} }
|
||||
|
||||
// Variance is a no-op.
|
||||
func (NilSample) Variance() float64 { return 0.0 }
|
||||
|
||||
// SampleMax returns the maximum value of the slice of int64.
|
||||
func SampleMax(values []int64) int64 {
|
||||
if 0 == len(values) {
|
||||
return 0
|
||||
}
|
||||
var max int64 = math.MinInt64
|
||||
for _, v := range values {
|
||||
if max < v {
|
||||
max = v
|
||||
}
|
||||
}
|
||||
return max
|
||||
}
|
||||
|
||||
// SampleMean returns the mean value of the slice of int64.
|
||||
func SampleMean(values []int64) float64 {
|
||||
if 0 == len(values) {
|
||||
return 0.0
|
||||
}
|
||||
return float64(SampleSum(values)) / float64(len(values))
|
||||
}
|
||||
|
||||
// SampleMin returns the minimum value of the slice of int64.
|
||||
func SampleMin(values []int64) int64 {
|
||||
if 0 == len(values) {
|
||||
return 0
|
||||
}
|
||||
var min int64 = math.MaxInt64
|
||||
for _, v := range values {
|
||||
if min > v {
|
||||
min = v
|
||||
}
|
||||
}
|
||||
return min
|
||||
}
|
||||
|
||||
// SamplePercentiles returns an arbitrary percentile of the slice of int64.
|
||||
func SamplePercentile(values int64Slice, p float64) float64 {
|
||||
return SamplePercentiles(values, []float64{p})[0]
|
||||
}
|
||||
|
||||
// SamplePercentiles returns a slice of arbitrary percentiles of the slice of
|
||||
// int64.
|
||||
func SamplePercentiles(values int64Slice, ps []float64) []float64 {
|
||||
scores := make([]float64, len(ps))
|
||||
size := len(values)
|
||||
if size > 0 {
|
||||
sort.Sort(values)
|
||||
for i, p := range ps {
|
||||
pos := p * float64(size+1)
|
||||
if pos < 1.0 {
|
||||
scores[i] = float64(values[0])
|
||||
} else if pos >= float64(size) {
|
||||
scores[i] = float64(values[size-1])
|
||||
} else {
|
||||
lower := float64(values[int(pos)-1])
|
||||
upper := float64(values[int(pos)])
|
||||
scores[i] = lower + (pos-math.Floor(pos))*(upper-lower)
|
||||
}
|
||||
}
|
||||
}
|
||||
return scores
|
||||
}
|
||||
|
||||
// SampleSnapshot is a read-only copy of another Sample.
|
||||
type SampleSnapshot struct {
|
||||
count int64
|
||||
values []int64
|
||||
}
|
||||
|
||||
// Clear panics.
|
||||
func (*SampleSnapshot) Clear() {
|
||||
panic("Clear called on a SampleSnapshot")
|
||||
}
|
||||
|
||||
// Count returns the count of inputs at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Count() int64 { return s.count }
|
||||
|
||||
// Max returns the maximal value at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Max() int64 { return SampleMax(s.values) }
|
||||
|
||||
// Mean returns the mean value at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Mean() float64 { return SampleMean(s.values) }
|
||||
|
||||
// Min returns the minimal value at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Min() int64 { return SampleMin(s.values) }
|
||||
|
||||
// Percentile returns an arbitrary percentile of values at the time the
|
||||
// snapshot was taken.
|
||||
func (s *SampleSnapshot) Percentile(p float64) float64 {
|
||||
return SamplePercentile(s.values, p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of values at the time
|
||||
// the snapshot was taken.
|
||||
func (s *SampleSnapshot) Percentiles(ps []float64) []float64 {
|
||||
return SamplePercentiles(s.values, ps)
|
||||
}
|
||||
|
||||
// Size returns the size of the sample at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Size() int { return len(s.values) }
|
||||
|
||||
// Snapshot returns the snapshot.
|
||||
func (s *SampleSnapshot) Snapshot() Sample { return s }
|
||||
|
||||
// StdDev returns the standard deviation of values at the time the snapshot was
|
||||
// taken.
|
||||
func (s *SampleSnapshot) StdDev() float64 { return SampleStdDev(s.values) }
|
||||
|
||||
// Sum returns the sum of values at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Sum() int64 { return SampleSum(s.values) }
|
||||
|
||||
// Update panics.
|
||||
func (*SampleSnapshot) Update(int64) {
|
||||
panic("Update called on a SampleSnapshot")
|
||||
}
|
||||
|
||||
// Values returns a copy of the values in the sample.
|
||||
func (s *SampleSnapshot) Values() []int64 {
|
||||
values := make([]int64, len(s.values))
|
||||
copy(values, s.values)
|
||||
return values
|
||||
}
|
||||
|
||||
// Variance returns the variance of values at the time the snapshot was taken.
|
||||
func (s *SampleSnapshot) Variance() float64 { return SampleVariance(s.values) }
|
||||
|
||||
// SampleStdDev returns the standard deviation of the slice of int64.
|
||||
func SampleStdDev(values []int64) float64 {
|
||||
return math.Sqrt(SampleVariance(values))
|
||||
}
|
||||
|
||||
// SampleSum returns the sum of the slice of int64.
|
||||
func SampleSum(values []int64) int64 {
|
||||
var sum int64
|
||||
for _, v := range values {
|
||||
sum += v
|
||||
}
|
||||
return sum
|
||||
}
|
||||
|
||||
// SampleVariance returns the variance of the slice of int64.
|
||||
func SampleVariance(values []int64) float64 {
|
||||
if 0 == len(values) {
|
||||
return 0.0
|
||||
}
|
||||
m := SampleMean(values)
|
||||
var sum float64
|
||||
for _, v := range values {
|
||||
d := float64(v) - m
|
||||
sum += d * d
|
||||
}
|
||||
return sum / float64(len(values))
|
||||
}
|
||||
|
||||
// A uniform sample using Vitter's Algorithm R.
|
||||
//
|
||||
// <http://www.cs.umd.edu/~samir/498/vitter.pdf>
|
||||
type UniformSample struct {
|
||||
count int64
|
||||
mutex sync.Mutex
|
||||
reservoirSize int
|
||||
values []int64
|
||||
}
|
||||
|
||||
// NewUniformSample constructs a new uniform sample with the given reservoir
|
||||
// size.
|
||||
func NewUniformSample(reservoirSize int) Sample {
|
||||
return &UniformSample{
|
||||
reservoirSize: reservoirSize,
|
||||
values: make([]int64, 0, reservoirSize),
|
||||
}
|
||||
}
|
||||
|
||||
// Clear clears all samples.
|
||||
func (s *UniformSample) Clear() {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
s.count = 0
|
||||
s.values = make([]int64, 0, s.reservoirSize)
|
||||
}
|
||||
|
||||
// Count returns the number of samples recorded, which may exceed the
|
||||
// reservoir size.
|
||||
func (s *UniformSample) Count() int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return s.count
|
||||
}
|
||||
|
||||
// Max returns the maximum value in the sample, which may not be the maximum
|
||||
// value ever to be part of the sample.
|
||||
func (s *UniformSample) Max() int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SampleMax(s.values)
|
||||
}
|
||||
|
||||
// Mean returns the mean of the values in the sample.
|
||||
func (s *UniformSample) Mean() float64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SampleMean(s.values)
|
||||
}
|
||||
|
||||
// Min returns the minimum value in the sample, which may not be the minimum
|
||||
// value ever to be part of the sample.
|
||||
func (s *UniformSample) Min() int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SampleMin(s.values)
|
||||
}
|
||||
|
||||
// Percentile returns an arbitrary percentile of values in the sample.
|
||||
func (s *UniformSample) Percentile(p float64) float64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SamplePercentile(s.values, p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of values in the
|
||||
// sample.
|
||||
func (s *UniformSample) Percentiles(ps []float64) []float64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SamplePercentiles(s.values, ps)
|
||||
}
|
||||
|
||||
// Size returns the size of the sample, which is at most the reservoir size.
|
||||
func (s *UniformSample) Size() int {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return len(s.values)
|
||||
}
|
||||
|
||||
// Snapshot returns a read-only copy of the sample.
|
||||
func (s *UniformSample) Snapshot() Sample {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
values := make([]int64, len(s.values))
|
||||
copy(values, s.values)
|
||||
return &SampleSnapshot{
|
||||
count: s.count,
|
||||
values: values,
|
||||
}
|
||||
}
|
||||
|
||||
// StdDev returns the standard deviation of the values in the sample.
|
||||
func (s *UniformSample) StdDev() float64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SampleStdDev(s.values)
|
||||
}
|
||||
|
||||
// Sum returns the sum of the values in the sample.
|
||||
func (s *UniformSample) Sum() int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SampleSum(s.values)
|
||||
}
|
||||
|
||||
// Update samples a new value.
|
||||
func (s *UniformSample) Update(v int64) {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
s.count++
|
||||
if len(s.values) < s.reservoirSize {
|
||||
s.values = append(s.values, v)
|
||||
} else {
|
||||
r := rand.Int63n(s.count)
|
||||
if r < int64(len(s.values)) {
|
||||
s.values[int(r)] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Values returns a copy of the values in the sample.
|
||||
func (s *UniformSample) Values() []int64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
values := make([]int64, len(s.values))
|
||||
copy(values, s.values)
|
||||
return values
|
||||
}
|
||||
|
||||
// Variance returns the variance of the values in the sample.
|
||||
func (s *UniformSample) Variance() float64 {
|
||||
s.mutex.Lock()
|
||||
defer s.mutex.Unlock()
|
||||
return SampleVariance(s.values)
|
||||
}
|
||||
|
||||
// expDecaySample represents an individual sample in a heap.
|
||||
type expDecaySample struct {
|
||||
k float64
|
||||
v int64
|
||||
}
|
||||
|
||||
func newExpDecaySampleHeap(reservoirSize int) *expDecaySampleHeap {
|
||||
return &expDecaySampleHeap{make([]expDecaySample, 0, reservoirSize)}
|
||||
}
|
||||
|
||||
// expDecaySampleHeap is a min-heap of expDecaySamples.
|
||||
// The internal implementation is copied from the standard library's container/heap
|
||||
type expDecaySampleHeap struct {
|
||||
s []expDecaySample
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) Clear() {
|
||||
h.s = h.s[:0]
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) Push(s expDecaySample) {
|
||||
n := len(h.s)
|
||||
h.s = h.s[0 : n+1]
|
||||
h.s[n] = s
|
||||
h.up(n)
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) Pop() expDecaySample {
|
||||
n := len(h.s) - 1
|
||||
h.s[0], h.s[n] = h.s[n], h.s[0]
|
||||
h.down(0, n)
|
||||
|
||||
n = len(h.s)
|
||||
s := h.s[n-1]
|
||||
h.s = h.s[0 : n-1]
|
||||
return s
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) Size() int {
|
||||
return len(h.s)
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) Values() []expDecaySample {
|
||||
return h.s
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) up(j int) {
|
||||
for {
|
||||
i := (j - 1) / 2 // parent
|
||||
if i == j || !(h.s[j].k < h.s[i].k) {
|
||||
break
|
||||
}
|
||||
h.s[i], h.s[j] = h.s[j], h.s[i]
|
||||
j = i
|
||||
}
|
||||
}
|
||||
|
||||
func (h *expDecaySampleHeap) down(i, n int) {
|
||||
for {
|
||||
j1 := 2*i + 1
|
||||
if j1 >= n || j1 < 0 { // j1 < 0 after int overflow
|
||||
break
|
||||
}
|
||||
j := j1 // left child
|
||||
if j2 := j1 + 1; j2 < n && !(h.s[j1].k < h.s[j2].k) {
|
||||
j = j2 // = 2*i + 2 // right child
|
||||
}
|
||||
if !(h.s[j].k < h.s[i].k) {
|
||||
break
|
||||
}
|
||||
h.s[i], h.s[j] = h.s[j], h.s[i]
|
||||
i = j
|
||||
}
|
||||
}
|
||||
|
||||
type int64Slice []int64
|
||||
|
||||
func (p int64Slice) Len() int { return len(p) }
|
||||
func (p int64Slice) Less(i, j int) bool { return p[i] < p[j] }
|
||||
func (p int64Slice) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
|
||||
@@ -1,367 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"math/rand"
|
||||
"runtime"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Benchmark{Compute,Copy}{1000,1000000} demonstrate that, even for relatively
|
||||
// expensive computations like Variance, the cost of copying the Sample, as
|
||||
// approximated by a make and copy, is much greater than the cost of the
|
||||
// computation for small samples and only slightly less for large samples.
|
||||
func BenchmarkCompute1000(b *testing.B) {
|
||||
s := make([]int64, 1000)
|
||||
for i := 0; i < len(s); i++ {
|
||||
s[i] = int64(i)
|
||||
}
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
SampleVariance(s)
|
||||
}
|
||||
}
|
||||
func BenchmarkCompute1000000(b *testing.B) {
|
||||
s := make([]int64, 1000000)
|
||||
for i := 0; i < len(s); i++ {
|
||||
s[i] = int64(i)
|
||||
}
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
SampleVariance(s)
|
||||
}
|
||||
}
|
||||
func BenchmarkCopy1000(b *testing.B) {
|
||||
s := make([]int64, 1000)
|
||||
for i := 0; i < len(s); i++ {
|
||||
s[i] = int64(i)
|
||||
}
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
sCopy := make([]int64, len(s))
|
||||
copy(sCopy, s)
|
||||
}
|
||||
}
|
||||
func BenchmarkCopy1000000(b *testing.B) {
|
||||
s := make([]int64, 1000000)
|
||||
for i := 0; i < len(s); i++ {
|
||||
s[i] = int64(i)
|
||||
}
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
sCopy := make([]int64, len(s))
|
||||
copy(sCopy, s)
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkExpDecaySample257(b *testing.B) {
|
||||
benchmarkSample(b, NewExpDecaySample(257, 0.015))
|
||||
}
|
||||
|
||||
func BenchmarkExpDecaySample514(b *testing.B) {
|
||||
benchmarkSample(b, NewExpDecaySample(514, 0.015))
|
||||
}
|
||||
|
||||
func BenchmarkExpDecaySample1028(b *testing.B) {
|
||||
benchmarkSample(b, NewExpDecaySample(1028, 0.015))
|
||||
}
|
||||
|
||||
func BenchmarkUniformSample257(b *testing.B) {
|
||||
benchmarkSample(b, NewUniformSample(257))
|
||||
}
|
||||
|
||||
func BenchmarkUniformSample514(b *testing.B) {
|
||||
benchmarkSample(b, NewUniformSample(514))
|
||||
}
|
||||
|
||||
func BenchmarkUniformSample1028(b *testing.B) {
|
||||
benchmarkSample(b, NewUniformSample(1028))
|
||||
}
|
||||
|
||||
func TestExpDecaySample10(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewExpDecaySample(100, 0.99)
|
||||
for i := 0; i < 10; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
if size := s.Count(); 10 != size {
|
||||
t.Errorf("s.Count(): 10 != %v\n", size)
|
||||
}
|
||||
if size := s.Size(); 10 != size {
|
||||
t.Errorf("s.Size(): 10 != %v\n", size)
|
||||
}
|
||||
if l := len(s.Values()); 10 != l {
|
||||
t.Errorf("len(s.Values()): 10 != %v\n", l)
|
||||
}
|
||||
for _, v := range s.Values() {
|
||||
if v > 10 || v < 0 {
|
||||
t.Errorf("out of range [0, 10): %v\n", v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpDecaySample100(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewExpDecaySample(1000, 0.01)
|
||||
for i := 0; i < 100; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
if size := s.Count(); 100 != size {
|
||||
t.Errorf("s.Count(): 100 != %v\n", size)
|
||||
}
|
||||
if size := s.Size(); 100 != size {
|
||||
t.Errorf("s.Size(): 100 != %v\n", size)
|
||||
}
|
||||
if l := len(s.Values()); 100 != l {
|
||||
t.Errorf("len(s.Values()): 100 != %v\n", l)
|
||||
}
|
||||
for _, v := range s.Values() {
|
||||
if v > 100 || v < 0 {
|
||||
t.Errorf("out of range [0, 100): %v\n", v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpDecaySample1000(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewExpDecaySample(100, 0.99)
|
||||
for i := 0; i < 1000; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
if size := s.Count(); 1000 != size {
|
||||
t.Errorf("s.Count(): 1000 != %v\n", size)
|
||||
}
|
||||
if size := s.Size(); 100 != size {
|
||||
t.Errorf("s.Size(): 100 != %v\n", size)
|
||||
}
|
||||
if l := len(s.Values()); 100 != l {
|
||||
t.Errorf("len(s.Values()): 100 != %v\n", l)
|
||||
}
|
||||
for _, v := range s.Values() {
|
||||
if v > 1000 || v < 0 {
|
||||
t.Errorf("out of range [0, 1000): %v\n", v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// This test makes sure that the sample's priority is not amplified by using
|
||||
// nanosecond duration since start rather than second duration since start.
|
||||
// The priority becomes +Inf quickly after starting if this is done,
|
||||
// effectively freezing the set of samples until a rescale step happens.
|
||||
func TestExpDecaySampleNanosecondRegression(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewExpDecaySample(100, 0.99)
|
||||
for i := 0; i < 100; i++ {
|
||||
s.Update(10)
|
||||
}
|
||||
time.Sleep(1 * time.Millisecond)
|
||||
for i := 0; i < 100; i++ {
|
||||
s.Update(20)
|
||||
}
|
||||
v := s.Values()
|
||||
avg := float64(0)
|
||||
for i := 0; i < len(v); i++ {
|
||||
avg += float64(v[i])
|
||||
}
|
||||
avg /= float64(len(v))
|
||||
if avg > 16 || avg < 14 {
|
||||
t.Errorf("out of range [14, 16]: %v\n", avg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpDecaySampleRescale(t *testing.T) {
|
||||
s := NewExpDecaySample(2, 0.001).(*ExpDecaySample)
|
||||
s.update(time.Now(), 1)
|
||||
s.update(time.Now().Add(time.Hour+time.Microsecond), 1)
|
||||
for _, v := range s.values.Values() {
|
||||
if v.k == 0.0 {
|
||||
t.Fatal("v.k == 0.0")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpDecaySampleSnapshot(t *testing.T) {
|
||||
now := time.Now()
|
||||
rand.Seed(1)
|
||||
s := NewExpDecaySample(100, 0.99)
|
||||
for i := 1; i <= 10000; i++ {
|
||||
s.(*ExpDecaySample).update(now.Add(time.Duration(i)), int64(i))
|
||||
}
|
||||
snapshot := s.Snapshot()
|
||||
s.Update(1)
|
||||
testExpDecaySampleStatistics(t, snapshot)
|
||||
}
|
||||
|
||||
func TestExpDecaySampleStatistics(t *testing.T) {
|
||||
now := time.Now()
|
||||
rand.Seed(1)
|
||||
s := NewExpDecaySample(100, 0.99)
|
||||
for i := 1; i <= 10000; i++ {
|
||||
s.(*ExpDecaySample).update(now.Add(time.Duration(i)), int64(i))
|
||||
}
|
||||
testExpDecaySampleStatistics(t, s)
|
||||
}
|
||||
|
||||
func TestUniformSample(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewUniformSample(100)
|
||||
for i := 0; i < 1000; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
if size := s.Count(); 1000 != size {
|
||||
t.Errorf("s.Count(): 1000 != %v\n", size)
|
||||
}
|
||||
if size := s.Size(); 100 != size {
|
||||
t.Errorf("s.Size(): 100 != %v\n", size)
|
||||
}
|
||||
if l := len(s.Values()); 100 != l {
|
||||
t.Errorf("len(s.Values()): 100 != %v\n", l)
|
||||
}
|
||||
for _, v := range s.Values() {
|
||||
if v > 1000 || v < 0 {
|
||||
t.Errorf("out of range [0, 100): %v\n", v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestUniformSampleIncludesTail(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewUniformSample(100)
|
||||
max := 100
|
||||
for i := 0; i < max; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
v := s.Values()
|
||||
sum := 0
|
||||
exp := (max - 1) * max / 2
|
||||
for i := 0; i < len(v); i++ {
|
||||
sum += int(v[i])
|
||||
}
|
||||
if exp != sum {
|
||||
t.Errorf("sum: %v != %v\n", exp, sum)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUniformSampleSnapshot(t *testing.T) {
|
||||
s := NewUniformSample(100)
|
||||
for i := 1; i <= 10000; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
snapshot := s.Snapshot()
|
||||
s.Update(1)
|
||||
testUniformSampleStatistics(t, snapshot)
|
||||
}
|
||||
|
||||
func TestUniformSampleStatistics(t *testing.T) {
|
||||
rand.Seed(1)
|
||||
s := NewUniformSample(100)
|
||||
for i := 1; i <= 10000; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
testUniformSampleStatistics(t, s)
|
||||
}
|
||||
|
||||
func benchmarkSample(b *testing.B, s Sample) {
|
||||
var memStats runtime.MemStats
|
||||
runtime.ReadMemStats(&memStats)
|
||||
pauseTotalNs := memStats.PauseTotalNs
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
s.Update(1)
|
||||
}
|
||||
b.StopTimer()
|
||||
runtime.GC()
|
||||
runtime.ReadMemStats(&memStats)
|
||||
b.Logf("GC cost: %d ns/op", int(memStats.PauseTotalNs-pauseTotalNs)/b.N)
|
||||
}
|
||||
|
||||
func testExpDecaySampleStatistics(t *testing.T, s Sample) {
|
||||
if count := s.Count(); 10000 != count {
|
||||
t.Errorf("s.Count(): 10000 != %v\n", count)
|
||||
}
|
||||
if min := s.Min(); 107 != min {
|
||||
t.Errorf("s.Min(): 107 != %v\n", min)
|
||||
}
|
||||
if max := s.Max(); 10000 != max {
|
||||
t.Errorf("s.Max(): 10000 != %v\n", max)
|
||||
}
|
||||
if mean := s.Mean(); 4965.98 != mean {
|
||||
t.Errorf("s.Mean(): 4965.98 != %v\n", mean)
|
||||
}
|
||||
if stdDev := s.StdDev(); 2959.825156930727 != stdDev {
|
||||
t.Errorf("s.StdDev(): 2959.825156930727 != %v\n", stdDev)
|
||||
}
|
||||
ps := s.Percentiles([]float64{0.5, 0.75, 0.99})
|
||||
if 4615 != ps[0] {
|
||||
t.Errorf("median: 4615 != %v\n", ps[0])
|
||||
}
|
||||
if 7672 != ps[1] {
|
||||
t.Errorf("75th percentile: 7672 != %v\n", ps[1])
|
||||
}
|
||||
if 9998.99 != ps[2] {
|
||||
t.Errorf("99th percentile: 9998.99 != %v\n", ps[2])
|
||||
}
|
||||
}
|
||||
|
||||
func testUniformSampleStatistics(t *testing.T, s Sample) {
|
||||
if count := s.Count(); 10000 != count {
|
||||
t.Errorf("s.Count(): 10000 != %v\n", count)
|
||||
}
|
||||
if min := s.Min(); 37 != min {
|
||||
t.Errorf("s.Min(): 37 != %v\n", min)
|
||||
}
|
||||
if max := s.Max(); 9989 != max {
|
||||
t.Errorf("s.Max(): 9989 != %v\n", max)
|
||||
}
|
||||
if mean := s.Mean(); 4748.14 != mean {
|
||||
t.Errorf("s.Mean(): 4748.14 != %v\n", mean)
|
||||
}
|
||||
if stdDev := s.StdDev(); 2826.684117548333 != stdDev {
|
||||
t.Errorf("s.StdDev(): 2826.684117548333 != %v\n", stdDev)
|
||||
}
|
||||
ps := s.Percentiles([]float64{0.5, 0.75, 0.99})
|
||||
if 4599 != ps[0] {
|
||||
t.Errorf("median: 4599 != %v\n", ps[0])
|
||||
}
|
||||
if 7380.5 != ps[1] {
|
||||
t.Errorf("75th percentile: 7380.5 != %v\n", ps[1])
|
||||
}
|
||||
if 9986.429999999998 != ps[2] {
|
||||
t.Errorf("99th percentile: 9986.429999999998 != %v\n", ps[2])
|
||||
}
|
||||
}
|
||||
|
||||
// TestUniformSampleConcurrentUpdateCount would expose data race problems with
|
||||
// concurrent Update and Count calls on Sample when test is called with -race
|
||||
// argument
|
||||
func TestUniformSampleConcurrentUpdateCount(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skipping in short mode")
|
||||
}
|
||||
s := NewUniformSample(100)
|
||||
for i := 0; i < 100; i++ {
|
||||
s.Update(int64(i))
|
||||
}
|
||||
quit := make(chan struct{})
|
||||
go func() {
|
||||
t := time.NewTicker(10 * time.Millisecond)
|
||||
for {
|
||||
select {
|
||||
case <-t.C:
|
||||
s.Update(rand.Int63())
|
||||
case <-quit:
|
||||
t.Stop()
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
for i := 0; i < 1000; i++ {
|
||||
s.Count()
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
quit <- struct{}{}
|
||||
}
|
||||
+53
-17
@@ -1,25 +1,27 @@
|
||||
package metrics
|
||||
|
||||
import "github.com/grafana/grafana/pkg/setting"
|
||||
import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
type MetricPublisher interface {
|
||||
Publish(metrics []Metric)
|
||||
}
|
||||
"github.com/grafana/grafana/pkg/metrics/graphitebridge"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
ini "gopkg.in/ini.v1"
|
||||
)
|
||||
|
||||
type MetricSettings struct {
|
||||
Enabled bool
|
||||
IntervalSeconds int64
|
||||
|
||||
Publishers []MetricPublisher
|
||||
Enabled bool
|
||||
IntervalSeconds int64
|
||||
GraphiteBridgeConfig *graphitebridge.Config
|
||||
}
|
||||
|
||||
func readSettings() *MetricSettings {
|
||||
func ReadSettings(file *ini.File) *MetricSettings {
|
||||
var settings = &MetricSettings{
|
||||
Enabled: false,
|
||||
Publishers: make([]MetricPublisher, 0),
|
||||
Enabled: false,
|
||||
}
|
||||
|
||||
var section, err = setting.Cfg.GetSection("metrics")
|
||||
var section, err = file.GetSection("metrics")
|
||||
if err != nil {
|
||||
metricsLogger.Crit("Unable to find metrics config section", "error", err)
|
||||
return nil
|
||||
@@ -32,12 +34,46 @@ func readSettings() *MetricSettings {
|
||||
return settings
|
||||
}
|
||||
|
||||
if graphitePublisher, err := CreateGraphitePublisher(); err != nil {
|
||||
metricsLogger.Error("Failed to init Graphite metric publisher", "error", err)
|
||||
} else if graphitePublisher != nil {
|
||||
metricsLogger.Info("Metrics publisher initialized", "type", "graphite")
|
||||
settings.Publishers = append(settings.Publishers, graphitePublisher)
|
||||
cfg, err := parseGraphiteSettings(settings, file)
|
||||
if err != nil {
|
||||
metricsLogger.Crit("Unable to parse metrics graphite section", "error", err)
|
||||
return nil
|
||||
}
|
||||
|
||||
settings.GraphiteBridgeConfig = cfg
|
||||
|
||||
return settings
|
||||
}
|
||||
|
||||
func parseGraphiteSettings(settings *MetricSettings, file *ini.File) (*graphitebridge.Config, error) {
|
||||
graphiteSection, err := setting.Cfg.GetSection("metrics.graphite")
|
||||
if err != nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
address := graphiteSection.Key("address").String()
|
||||
if address == "" {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
cfg := &graphitebridge.Config{
|
||||
URL: address,
|
||||
Prefix: graphiteSection.Key("prefix").MustString("prod.grafana.%(instance_name)s"),
|
||||
CountersAsDelta: true,
|
||||
Gatherer: prometheus.DefaultGatherer,
|
||||
Interval: time.Duration(settings.IntervalSeconds) * time.Second,
|
||||
Timeout: 10 * time.Second,
|
||||
Logger: &logWrapper{logger: metricsLogger},
|
||||
ErrorHandling: graphitebridge.ContinueOnError,
|
||||
}
|
||||
|
||||
safeInstanceName := strings.Replace(setting.InstanceName, ".", "_", -1)
|
||||
prefix := graphiteSection.Key("prefix").Value()
|
||||
|
||||
if prefix == "" {
|
||||
prefix = "prod.grafana.%(instance_name)s."
|
||||
}
|
||||
|
||||
cfg.Prefix = strings.Replace(prefix, "%(instance_name)s", safeInstanceName, -1)
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
@@ -1,310 +0,0 @@
|
||||
// includes code from
|
||||
// https://raw.githubusercontent.com/rcrowley/go-metrics/master/sample.go
|
||||
// Copyright 2012 Richard Crowley. All rights reserved.
|
||||
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Timers capture the duration and rate of events.
|
||||
type Timer interface {
|
||||
Metric
|
||||
|
||||
Count() int64
|
||||
Max() int64
|
||||
Mean() float64
|
||||
Min() int64
|
||||
Percentile(float64) float64
|
||||
Percentiles([]float64) []float64
|
||||
Rate1() float64
|
||||
Rate5() float64
|
||||
Rate15() float64
|
||||
RateMean() float64
|
||||
StdDev() float64
|
||||
Sum() int64
|
||||
Time(func())
|
||||
Update(time.Duration)
|
||||
UpdateSince(time.Time)
|
||||
Variance() float64
|
||||
}
|
||||
|
||||
// NewCustomTimer constructs a new StandardTimer from a Histogram and a Meter.
|
||||
func NewCustomTimer(meta *MetricMeta, h Histogram, m Meter) Timer {
|
||||
if UseNilMetrics {
|
||||
return NilTimer{}
|
||||
}
|
||||
return &StandardTimer{
|
||||
MetricMeta: meta,
|
||||
histogram: h,
|
||||
meter: m,
|
||||
}
|
||||
}
|
||||
|
||||
// NewTimer constructs a new StandardTimer using an exponentially-decaying
|
||||
// sample with the same reservoir size and alpha as UNIX load averages.
|
||||
func NewTimer(meta *MetricMeta) Timer {
|
||||
if UseNilMetrics {
|
||||
return NilTimer{}
|
||||
}
|
||||
return &StandardTimer{
|
||||
MetricMeta: meta,
|
||||
histogram: NewHistogram(meta, NewExpDecaySample(1028, 0.015)),
|
||||
meter: NewMeter(meta),
|
||||
}
|
||||
}
|
||||
|
||||
func RegTimer(name string, tagStrings ...string) Timer {
|
||||
tr := NewTimer(NewMetricMeta(name, tagStrings))
|
||||
MetricStats.Register(tr)
|
||||
return tr
|
||||
}
|
||||
|
||||
// NilTimer is a no-op Timer.
|
||||
type NilTimer struct {
|
||||
*MetricMeta
|
||||
h Histogram
|
||||
m Meter
|
||||
}
|
||||
|
||||
// Count is a no-op.
|
||||
func (NilTimer) Count() int64 { return 0 }
|
||||
|
||||
// Max is a no-op.
|
||||
func (NilTimer) Max() int64 { return 0 }
|
||||
|
||||
// Mean is a no-op.
|
||||
func (NilTimer) Mean() float64 { return 0.0 }
|
||||
|
||||
// Min is a no-op.
|
||||
func (NilTimer) Min() int64 { return 0 }
|
||||
|
||||
// Percentile is a no-op.
|
||||
func (NilTimer) Percentile(p float64) float64 { return 0.0 }
|
||||
|
||||
// Percentiles is a no-op.
|
||||
func (NilTimer) Percentiles(ps []float64) []float64 {
|
||||
return make([]float64, len(ps))
|
||||
}
|
||||
|
||||
// Rate1 is a no-op.
|
||||
func (NilTimer) Rate1() float64 { return 0.0 }
|
||||
|
||||
// Rate5 is a no-op.
|
||||
func (NilTimer) Rate5() float64 { return 0.0 }
|
||||
|
||||
// Rate15 is a no-op.
|
||||
func (NilTimer) Rate15() float64 { return 0.0 }
|
||||
|
||||
// RateMean is a no-op.
|
||||
func (NilTimer) RateMean() float64 { return 0.0 }
|
||||
|
||||
// Snapshot is a no-op.
|
||||
func (n NilTimer) Snapshot() Metric { return n }
|
||||
|
||||
// StdDev is a no-op.
|
||||
func (NilTimer) StdDev() float64 { return 0.0 }
|
||||
|
||||
// Sum is a no-op.
|
||||
func (NilTimer) Sum() int64 { return 0 }
|
||||
|
||||
// Time is a no-op.
|
||||
func (NilTimer) Time(func()) {}
|
||||
|
||||
// Update is a no-op.
|
||||
func (NilTimer) Update(time.Duration) {}
|
||||
|
||||
// UpdateSince is a no-op.
|
||||
func (NilTimer) UpdateSince(time.Time) {}
|
||||
|
||||
// Variance is a no-op.
|
||||
func (NilTimer) Variance() float64 { return 0.0 }
|
||||
|
||||
// StandardTimer is the standard implementation of a Timer and uses a Histogram
|
||||
// and Meter.
|
||||
type StandardTimer struct {
|
||||
*MetricMeta
|
||||
histogram Histogram
|
||||
meter Meter
|
||||
mutex sync.Mutex
|
||||
}
|
||||
|
||||
// Count returns the number of events recorded.
|
||||
func (t *StandardTimer) Count() int64 {
|
||||
return t.histogram.Count()
|
||||
}
|
||||
|
||||
// Max returns the maximum value in the sample.
|
||||
func (t *StandardTimer) Max() int64 {
|
||||
return t.histogram.Max()
|
||||
}
|
||||
|
||||
// Mean returns the mean of the values in the sample.
|
||||
func (t *StandardTimer) Mean() float64 {
|
||||
return t.histogram.Mean()
|
||||
}
|
||||
|
||||
// Min returns the minimum value in the sample.
|
||||
func (t *StandardTimer) Min() int64 {
|
||||
return t.histogram.Min()
|
||||
}
|
||||
|
||||
// Percentile returns an arbitrary percentile of the values in the sample.
|
||||
func (t *StandardTimer) Percentile(p float64) float64 {
|
||||
return t.histogram.Percentile(p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of the values in the
|
||||
// sample.
|
||||
func (t *StandardTimer) Percentiles(ps []float64) []float64 {
|
||||
return t.histogram.Percentiles(ps)
|
||||
}
|
||||
|
||||
// Rate1 returns the one-minute moving average rate of events per second.
|
||||
func (t *StandardTimer) Rate1() float64 {
|
||||
return t.meter.Rate1()
|
||||
}
|
||||
|
||||
// Rate5 returns the five-minute moving average rate of events per second.
|
||||
func (t *StandardTimer) Rate5() float64 {
|
||||
return t.meter.Rate5()
|
||||
}
|
||||
|
||||
// Rate15 returns the fifteen-minute moving average rate of events per second.
|
||||
func (t *StandardTimer) Rate15() float64 {
|
||||
return t.meter.Rate15()
|
||||
}
|
||||
|
||||
// RateMean returns the meter's mean rate of events per second.
|
||||
func (t *StandardTimer) RateMean() float64 {
|
||||
return t.meter.RateMean()
|
||||
}
|
||||
|
||||
// Snapshot returns a read-only copy of the timer.
|
||||
func (t *StandardTimer) Snapshot() Metric {
|
||||
t.mutex.Lock()
|
||||
defer t.mutex.Unlock()
|
||||
return &TimerSnapshot{
|
||||
MetricMeta: t.MetricMeta,
|
||||
histogram: t.histogram.Snapshot().(*HistogramSnapshot),
|
||||
meter: t.meter.Snapshot().(*MeterSnapshot),
|
||||
}
|
||||
}
|
||||
|
||||
// StdDev returns the standard deviation of the values in the sample.
|
||||
func (t *StandardTimer) StdDev() float64 {
|
||||
return t.histogram.StdDev()
|
||||
}
|
||||
|
||||
// Sum returns the sum in the sample.
|
||||
func (t *StandardTimer) Sum() int64 {
|
||||
return t.histogram.Sum()
|
||||
}
|
||||
|
||||
// Record the duration of the execution of the given function.
|
||||
func (t *StandardTimer) Time(f func()) {
|
||||
ts := time.Now()
|
||||
f()
|
||||
t.Update(time.Since(ts))
|
||||
}
|
||||
|
||||
// Record the duration of an event.
|
||||
func (t *StandardTimer) Update(d time.Duration) {
|
||||
t.mutex.Lock()
|
||||
defer t.mutex.Unlock()
|
||||
t.histogram.Update(int64(d))
|
||||
t.meter.Mark(1)
|
||||
}
|
||||
|
||||
// Record the duration of an event that started at a time and ends now.
|
||||
func (t *StandardTimer) UpdateSince(ts time.Time) {
|
||||
t.mutex.Lock()
|
||||
defer t.mutex.Unlock()
|
||||
sinceMs := time.Since(ts) / time.Millisecond
|
||||
t.histogram.Update(int64(sinceMs))
|
||||
t.meter.Mark(1)
|
||||
}
|
||||
|
||||
// Variance returns the variance of the values in the sample.
|
||||
func (t *StandardTimer) Variance() float64 {
|
||||
return t.histogram.Variance()
|
||||
}
|
||||
|
||||
// TimerSnapshot is a read-only copy of another Timer.
|
||||
type TimerSnapshot struct {
|
||||
*MetricMeta
|
||||
histogram *HistogramSnapshot
|
||||
meter *MeterSnapshot
|
||||
}
|
||||
|
||||
// Count returns the number of events recorded at the time the snapshot was
|
||||
// taken.
|
||||
func (t *TimerSnapshot) Count() int64 { return t.histogram.Count() }
|
||||
|
||||
// Max returns the maximum value at the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Max() int64 { return t.histogram.Max() }
|
||||
|
||||
// Mean returns the mean value at the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Mean() float64 { return t.histogram.Mean() }
|
||||
|
||||
// Min returns the minimum value at the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Min() int64 { return t.histogram.Min() }
|
||||
|
||||
// Percentile returns an arbitrary percentile of sampled values at the time the
|
||||
// snapshot was taken.
|
||||
func (t *TimerSnapshot) Percentile(p float64) float64 {
|
||||
return t.histogram.Percentile(p)
|
||||
}
|
||||
|
||||
// Percentiles returns a slice of arbitrary percentiles of sampled values at
|
||||
// the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Percentiles(ps []float64) []float64 {
|
||||
return t.histogram.Percentiles(ps)
|
||||
}
|
||||
|
||||
// Rate1 returns the one-minute moving average rate of events per second at the
|
||||
// time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Rate1() float64 { return t.meter.Rate1() }
|
||||
|
||||
// Rate5 returns the five-minute moving average rate of events per second at
|
||||
// the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Rate5() float64 { return t.meter.Rate5() }
|
||||
|
||||
// Rate15 returns the fifteen-minute moving average rate of events per second
|
||||
// at the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Rate15() float64 { return t.meter.Rate15() }
|
||||
|
||||
// RateMean returns the meter's mean rate of events per second at the time the
|
||||
// snapshot was taken.
|
||||
func (t *TimerSnapshot) RateMean() float64 { return t.meter.RateMean() }
|
||||
|
||||
// Snapshot returns the snapshot.
|
||||
func (t *TimerSnapshot) Snapshot() Metric { return t }
|
||||
|
||||
// StdDev returns the standard deviation of the values at the time the snapshot
|
||||
// was taken.
|
||||
func (t *TimerSnapshot) StdDev() float64 { return t.histogram.StdDev() }
|
||||
|
||||
// Sum returns the sum at the time the snapshot was taken.
|
||||
func (t *TimerSnapshot) Sum() int64 { return t.histogram.Sum() }
|
||||
|
||||
// Time panics.
|
||||
func (*TimerSnapshot) Time(func()) {
|
||||
panic("Time called on a TimerSnapshot")
|
||||
}
|
||||
|
||||
// Update panics.
|
||||
func (*TimerSnapshot) Update(time.Duration) {
|
||||
panic("Update called on a TimerSnapshot")
|
||||
}
|
||||
|
||||
// UpdateSince panics.
|
||||
func (*TimerSnapshot) UpdateSince(time.Time) {
|
||||
panic("UpdateSince called on a TimerSnapshot")
|
||||
}
|
||||
|
||||
// Variance returns the variance of the values at the time the snapshot was
|
||||
// taken.
|
||||
func (t *TimerSnapshot) Variance() float64 { return t.histogram.Variance() }
|
||||
@@ -19,8 +19,8 @@ import (
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"gopkg.in/macaron.v1"
|
||||
)
|
||||
|
||||
@@ -35,8 +35,8 @@ func Logger() macaron.Handler {
|
||||
timeTakenMs := time.Since(start) / time.Millisecond
|
||||
|
||||
if timer, ok := c.Data["perfmon.timer"]; ok {
|
||||
timerTyped := timer.(metrics.Timer)
|
||||
timerTyped.Update(timeTakenMs)
|
||||
timerTyped := timer.(prometheus.Summary)
|
||||
timerTyped.Observe(float64(timeTakenMs))
|
||||
}
|
||||
|
||||
status := rw.Status()
|
||||
|
||||
@@ -10,10 +10,10 @@ import (
|
||||
"github.com/grafana/grafana/pkg/components/apikeygen"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
l "github.com/grafana/grafana/pkg/login"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
"github.com/grafana/grafana/pkg/util"
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
)
|
||||
|
||||
type Context struct {
|
||||
@@ -251,7 +251,7 @@ func (ctx *Context) HasHelpFlag(flag m.HelpFlags1) bool {
|
||||
return ctx.HelpFlags1.HasFlag(flag)
|
||||
}
|
||||
|
||||
func (ctx *Context) TimeRequest(timer metrics.Timer) {
|
||||
func (ctx *Context) TimeRequest(timer prometheus.Summary) {
|
||||
ctx.Data["perfmon.timer"] = timer
|
||||
}
|
||||
|
||||
|
||||
@@ -2,19 +2,27 @@ package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
"gopkg.in/macaron.v1"
|
||||
)
|
||||
|
||||
func RequestMetrics() macaron.Handler {
|
||||
func RequestMetrics(handler string) macaron.Handler {
|
||||
return func(res http.ResponseWriter, req *http.Request, c *macaron.Context) {
|
||||
rw := res.(macaron.ResponseWriter)
|
||||
now := time.Now()
|
||||
c.Next()
|
||||
|
||||
status := rw.Status()
|
||||
|
||||
code := sanitizeCode(status)
|
||||
method := sanitizeMethod(req.Method)
|
||||
metrics.M_Http_Request_Total.WithLabelValues(handler, code, method).Inc()
|
||||
metrics.M_Http_Request_Summary.WithLabelValues(handler, code, method).Observe(time.Since(now).Seconds())
|
||||
|
||||
if strings.HasPrefix(req.RequestURI, "/api/datasources/proxy") {
|
||||
countProxyRequests(status)
|
||||
} else if strings.HasPrefix(req.RequestURI, "/api/") {
|
||||
@@ -28,38 +36,165 @@ func RequestMetrics() macaron.Handler {
|
||||
func countApiRequests(status int) {
|
||||
switch status {
|
||||
case 200:
|
||||
metrics.M_Api_Status_200.Inc(1)
|
||||
metrics.M_Api_Status.WithLabelValues("200").Inc()
|
||||
case 404:
|
||||
metrics.M_Api_Status_404.Inc(1)
|
||||
metrics.M_Api_Status.WithLabelValues("404").Inc()
|
||||
case 500:
|
||||
metrics.M_Api_Status_500.Inc(1)
|
||||
metrics.M_Api_Status.WithLabelValues("500").Inc()
|
||||
default:
|
||||
metrics.M_Api_Status_Unknown.Inc(1)
|
||||
metrics.M_Api_Status.WithLabelValues("unknown").Inc()
|
||||
}
|
||||
}
|
||||
|
||||
func countPageRequests(status int) {
|
||||
switch status {
|
||||
case 200:
|
||||
metrics.M_Page_Status_200.Inc(1)
|
||||
metrics.M_Page_Status.WithLabelValues("200").Inc()
|
||||
case 404:
|
||||
metrics.M_Page_Status_404.Inc(1)
|
||||
metrics.M_Page_Status.WithLabelValues("404").Inc()
|
||||
case 500:
|
||||
metrics.M_Page_Status_500.Inc(1)
|
||||
metrics.M_Page_Status.WithLabelValues("500").Inc()
|
||||
default:
|
||||
metrics.M_Page_Status_Unknown.Inc(1)
|
||||
metrics.M_Page_Status.WithLabelValues("unknown").Inc()
|
||||
}
|
||||
}
|
||||
|
||||
func countProxyRequests(status int) {
|
||||
switch status {
|
||||
case 200:
|
||||
metrics.M_Proxy_Status_200.Inc(1)
|
||||
metrics.M_Proxy_Status.WithLabelValues("200").Inc()
|
||||
case 404:
|
||||
metrics.M_Proxy_Status_404.Inc(1)
|
||||
metrics.M_Proxy_Status.WithLabelValues("400").Inc()
|
||||
case 500:
|
||||
metrics.M_Proxy_Status_500.Inc(1)
|
||||
metrics.M_Proxy_Status.WithLabelValues("500").Inc()
|
||||
default:
|
||||
metrics.M_Proxy_Status_Unknown.Inc(1)
|
||||
metrics.M_Proxy_Status.WithLabelValues("unknown").Inc()
|
||||
}
|
||||
}
|
||||
|
||||
func sanitizeMethod(m string) string {
|
||||
switch m {
|
||||
case "GET", "get":
|
||||
return "get"
|
||||
case "PUT", "put":
|
||||
return "put"
|
||||
case "HEAD", "head":
|
||||
return "head"
|
||||
case "POST", "post":
|
||||
return "post"
|
||||
case "DELETE", "delete":
|
||||
return "delete"
|
||||
case "CONNECT", "connect":
|
||||
return "connect"
|
||||
case "OPTIONS", "options":
|
||||
return "options"
|
||||
case "NOTIFY", "notify":
|
||||
return "notify"
|
||||
default:
|
||||
return strings.ToLower(m)
|
||||
}
|
||||
}
|
||||
|
||||
// If the wrapped http.Handler has not set a status code, i.e. the value is
|
||||
// currently 0, santizeCode will return 200, for consistency with behavior in
|
||||
// the stdlib.
|
||||
func sanitizeCode(s int) string {
|
||||
switch s {
|
||||
case 100:
|
||||
return "100"
|
||||
case 101:
|
||||
return "101"
|
||||
|
||||
case 200, 0:
|
||||
return "200"
|
||||
case 201:
|
||||
return "201"
|
||||
case 202:
|
||||
return "202"
|
||||
case 203:
|
||||
return "203"
|
||||
case 204:
|
||||
return "204"
|
||||
case 205:
|
||||
return "205"
|
||||
case 206:
|
||||
return "206"
|
||||
|
||||
case 300:
|
||||
return "300"
|
||||
case 301:
|
||||
return "301"
|
||||
case 302:
|
||||
return "302"
|
||||
case 304:
|
||||
return "304"
|
||||
case 305:
|
||||
return "305"
|
||||
case 307:
|
||||
return "307"
|
||||
|
||||
case 400:
|
||||
return "400"
|
||||
case 401:
|
||||
return "401"
|
||||
case 402:
|
||||
return "402"
|
||||
case 403:
|
||||
return "403"
|
||||
case 404:
|
||||
return "404"
|
||||
case 405:
|
||||
return "405"
|
||||
case 406:
|
||||
return "406"
|
||||
case 407:
|
||||
return "407"
|
||||
case 408:
|
||||
return "408"
|
||||
case 409:
|
||||
return "409"
|
||||
case 410:
|
||||
return "410"
|
||||
case 411:
|
||||
return "411"
|
||||
case 412:
|
||||
return "412"
|
||||
case 413:
|
||||
return "413"
|
||||
case 414:
|
||||
return "414"
|
||||
case 415:
|
||||
return "415"
|
||||
case 416:
|
||||
return "416"
|
||||
case 417:
|
||||
return "417"
|
||||
case 418:
|
||||
return "418"
|
||||
|
||||
case 500:
|
||||
return "500"
|
||||
case 501:
|
||||
return "501"
|
||||
case 502:
|
||||
return "502"
|
||||
case 503:
|
||||
return "503"
|
||||
case 504:
|
||||
return "504"
|
||||
case 505:
|
||||
return "505"
|
||||
|
||||
case 428:
|
||||
return "428"
|
||||
case 429:
|
||||
return "429"
|
||||
case 431:
|
||||
return "431"
|
||||
case 511:
|
||||
return "511"
|
||||
|
||||
default:
|
||||
return strconv.Itoa(s)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,8 +63,8 @@ func (e *DefaultEvalHandler) Eval(context *EvalContext) {
|
||||
context.EndTime = time.Now()
|
||||
context.Rule.State = e.getNewState(context)
|
||||
|
||||
elapsedTime := context.EndTime.Sub(context.StartTime) / time.Millisecond
|
||||
metrics.M_Alerting_Execution_Time.Update(elapsedTime)
|
||||
elapsedTime := context.EndTime.Sub(context.StartTime).Seconds()
|
||||
metrics.M_Alerting_Execution_Time.Observe(elapsedTime)
|
||||
}
|
||||
|
||||
// This should be move into evalContext once its been refactored.
|
||||
|
||||
@@ -10,6 +10,8 @@ import (
|
||||
"github.com/grafana/grafana/pkg/components/imguploader"
|
||||
"github.com/grafana/grafana/pkg/components/renderer"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
)
|
||||
|
||||
@@ -66,6 +68,7 @@ func (n *notificationService) sendNotifications(context *EvalContext, notifiers
|
||||
for _, notifier := range notifiers {
|
||||
not := notifier //avoid updating scope variable in go routine
|
||||
n.log.Info("Sending notification", "type", not.GetType(), "id", not.GetNotifierId(), "isDefault", not.GetIsDefault())
|
||||
metrics.M_Alerting_Notification_Sent.WithLabelValues(not.GetType()).Inc()
|
||||
g.Go(func() error { return not.Notify(context) })
|
||||
}
|
||||
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -47,7 +46,6 @@ type DingDingNotifier struct {
|
||||
|
||||
func (this *DingDingNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Sending dingding")
|
||||
metrics.M_Alerting_Notification_Sent_DingDing.Inc(1)
|
||||
|
||||
messageUrl, err := evalContext.GetRuleUrl()
|
||||
if err != nil {
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
@@ -61,7 +60,6 @@ func NewEmailNotifier(model *m.AlertNotification) (alerting.Notifier, error) {
|
||||
|
||||
func (this *EmailNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Sending alert notification to", "addresses", this.Addresses)
|
||||
metrics.M_Alerting_Notification_Sent_Email.Inc(1)
|
||||
|
||||
ruleUrl, err := evalContext.GetRuleUrl()
|
||||
if err != nil {
|
||||
|
||||
@@ -2,12 +2,12 @@ package notifiers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/url"
|
||||
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
"net/url"
|
||||
)
|
||||
|
||||
func init() {
|
||||
@@ -53,7 +53,6 @@ type LineNotifier struct {
|
||||
|
||||
func (this *LineNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Executing line notification", "ruleId", evalContext.Rule.Id, "notification", this.Name)
|
||||
metrics.M_Alerting_Notification_Sent_LINE.Inc(1)
|
||||
|
||||
var err error
|
||||
switch evalContext.Rule.State {
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -65,7 +64,6 @@ type OpsGenieNotifier struct {
|
||||
}
|
||||
|
||||
func (this *OpsGenieNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
metrics.M_Alerting_Notification_Sent_OpsGenie.Inc(1)
|
||||
|
||||
var err error
|
||||
switch evalContext.Rule.State {
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -63,7 +62,6 @@ type PagerdutyNotifier struct {
|
||||
}
|
||||
|
||||
func (this *PagerdutyNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
metrics.M_Alerting_Notification_Sent_PagerDuty.Inc(1)
|
||||
|
||||
if evalContext.Rule.State == m.AlertStateOK && !this.AutoResolve {
|
||||
this.log.Info("Not sending a trigger to Pagerduty", "state", evalContext.Rule.State, "auto resolve", this.AutoResolve)
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -125,7 +124,6 @@ type PushoverNotifier struct {
|
||||
}
|
||||
|
||||
func (this *PushoverNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
metrics.M_Alerting_Notification_Sent_Pushover.Inc(1)
|
||||
ruleUrl, err := evalContext.GetRuleUrl()
|
||||
if err != nil {
|
||||
this.log.Error("Failed get rule link", "error", err)
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -74,7 +73,6 @@ type SensuNotifier struct {
|
||||
|
||||
func (this *SensuNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Sending sensu result")
|
||||
metrics.M_Alerting_Notification_Sent_Sensu.Inc(1)
|
||||
|
||||
bodyJSON := simplejson.New()
|
||||
bodyJSON.Set("ruleId", evalContext.Rule.Id)
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
@@ -79,7 +78,6 @@ type SlackNotifier struct {
|
||||
|
||||
func (this *SlackNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Executing slack notification", "ruleId", evalContext.Rule.Id, "notification", this.Name)
|
||||
metrics.M_Alerting_Notification_Sent_Slack.Inc(1)
|
||||
|
||||
ruleUrl, err := evalContext.GetRuleUrl()
|
||||
if err != nil {
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -80,7 +79,6 @@ func NewTelegramNotifier(model *m.AlertNotification) (alerting.Notifier, error)
|
||||
func (this *TelegramNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Sending alert notification to", "bot_token", this.BotToken)
|
||||
this.log.Info("Sending alert notification to", "chat_id", this.ChatID)
|
||||
metrics.M_Alerting_Notification_Sent_Telegram.Inc(1)
|
||||
|
||||
bodyJSON := simplejson.New()
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -118,7 +117,6 @@ func NewThreemaNotifier(model *m.AlertNotification) (alerting.Notifier, error) {
|
||||
func (notifier *ThreemaNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
notifier.log.Info("Sending alert notification from", "threema_id", notifier.GatewayID)
|
||||
notifier.log.Info("Sending alert notification to", "threema_id", notifier.RecipientID)
|
||||
metrics.M_Alerting_Notification_Sent_Threema.Inc(1)
|
||||
|
||||
// Set up basic API request data
|
||||
data := url.Values{}
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
"github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
"github.com/grafana/grafana/pkg/setting"
|
||||
@@ -72,7 +71,6 @@ type VictoropsNotifier struct {
|
||||
// Notify sends notification to Victorops via POST to URL endpoint
|
||||
func (this *VictoropsNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Executing victorops notification", "ruleId", evalContext.Rule.Id, "notification", this.Name)
|
||||
metrics.M_Alerting_Notification_Sent_Victorops.Inc(1)
|
||||
|
||||
ruleUrl, err := evalContext.GetRuleUrl()
|
||||
if err != nil {
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"github.com/grafana/grafana/pkg/bus"
|
||||
"github.com/grafana/grafana/pkg/components/simplejson"
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/metrics"
|
||||
m "github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/services/alerting"
|
||||
)
|
||||
@@ -68,7 +67,6 @@ type WebhookNotifier struct {
|
||||
|
||||
func (this *WebhookNotifier) Notify(evalContext *alerting.EvalContext) error {
|
||||
this.log.Info("Sending webhook")
|
||||
metrics.M_Alerting_Notification_Sent_Webhook.Inc(1)
|
||||
|
||||
bodyJSON := simplejson.New()
|
||||
bodyJSON.Set("title", evalContext.GetNotificationTitle())
|
||||
|
||||
@@ -59,7 +59,7 @@ func (arr *DefaultRuleReader) Fetch() []*Rule {
|
||||
}
|
||||
}
|
||||
|
||||
metrics.M_Alerting_Active_Alerts.Update(int64(len(res)))
|
||||
metrics.M_Alerting_Active_Alerts.Set(float64(len(res)))
|
||||
return res
|
||||
}
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ func (handler *DefaultResultHandler) Handle(evalContext *EvalContext) error {
|
||||
annotationData.Set("noData", true)
|
||||
}
|
||||
|
||||
countStateResult(evalContext.Rule.State)
|
||||
metrics.M_Alerting_Result_State.WithLabelValues(string(evalContext.Rule.State)).Inc()
|
||||
if evalContext.ShouldUpdateAlertState() {
|
||||
handler.log.Info("New state change", "alertId", evalContext.Rule.Id, "newState", evalContext.Rule.State, "prev state", evalContext.PrevAlertState)
|
||||
|
||||
@@ -95,18 +95,3 @@ func (handler *DefaultResultHandler) Handle(evalContext *EvalContext) error {
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func countStateResult(state m.AlertStateType) {
|
||||
switch state {
|
||||
case m.AlertStatePending:
|
||||
metrics.M_Alerting_Result_State_Pending.Inc(1)
|
||||
case m.AlertStateAlerting:
|
||||
metrics.M_Alerting_Result_State_Alerting.Inc(1)
|
||||
case m.AlertStateOK:
|
||||
metrics.M_Alerting_Result_State_Ok.Inc(1)
|
||||
case m.AlertStatePaused:
|
||||
metrics.M_Alerting_Result_State_Paused.Inc(1)
|
||||
case m.AlertStateNoData:
|
||||
metrics.M_Alerting_Result_State_NoData.Inc(1)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -75,7 +75,7 @@ func SaveDashboard(cmd *m.SaveDashboardCommand) error {
|
||||
|
||||
if dash.Id == 0 {
|
||||
dash.Version = 1
|
||||
metrics.M_Models_Dashboard_Insert.Inc(1)
|
||||
metrics.M_Api_Dashboard_Insert.Inc()
|
||||
dash.Data.Set("version", dash.Version)
|
||||
affectedRows, err = sess.Insert(dash)
|
||||
} else {
|
||||
|
||||
@@ -20,7 +20,7 @@ func init() {
|
||||
}
|
||||
|
||||
func GetDataSourceById(query *m.GetDataSourceByIdQuery) error {
|
||||
metrics.M_DB_DataSource_QueryById.Inc(1)
|
||||
metrics.M_DB_DataSource_QueryById.Inc()
|
||||
|
||||
datasource := m.DataSource{OrgId: query.OrgId, Id: query.Id}
|
||||
has, err := x.Get(&datasource)
|
||||
|
||||
@@ -13,8 +13,11 @@ import (
|
||||
"github.com/grafana/grafana/pkg/log"
|
||||
"github.com/grafana/grafana/pkg/models"
|
||||
"github.com/grafana/grafana/pkg/tsdb"
|
||||
"github.com/prometheus/client_golang/api/prometheus"
|
||||
pmodel "github.com/prometheus/common/model"
|
||||
api "github.com/prometheus/client_golang/api"
|
||||
apiv1 "github.com/prometheus/client_golang/api/prometheus/v1"
|
||||
"github.com/prometheus/common/model"
|
||||
//api "github.com/prometheus/client_golang/api"
|
||||
//apiv1 "github.com/prometheus/client_golang/api/prometheus/v1"
|
||||
)
|
||||
|
||||
type PrometheusExecutor struct {
|
||||
@@ -57,26 +60,26 @@ func init() {
|
||||
legendFormat = regexp.MustCompile(`\{\{\s*(.+?)\s*\}\}`)
|
||||
}
|
||||
|
||||
func (e *PrometheusExecutor) getClient() (prometheus.QueryAPI, error) {
|
||||
cfg := prometheus.Config{
|
||||
Address: e.DataSource.Url,
|
||||
Transport: e.Transport,
|
||||
func (e *PrometheusExecutor) getClient() (apiv1.API, error) {
|
||||
cfg := api.Config{
|
||||
Address: e.DataSource.Url,
|
||||
RoundTripper: e.Transport,
|
||||
}
|
||||
|
||||
if e.BasicAuth {
|
||||
cfg.Transport = basicAuthTransport{
|
||||
cfg.RoundTripper = basicAuthTransport{
|
||||
Transport: e.Transport,
|
||||
username: e.BasicAuthUser,
|
||||
password: e.BasicAuthPassword,
|
||||
}
|
||||
}
|
||||
|
||||
client, err := prometheus.New(cfg)
|
||||
client, err := api.NewClient(cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return prometheus.NewQueryAPI(client), nil
|
||||
return apiv1.NewAPI(client), nil
|
||||
}
|
||||
|
||||
func (e *PrometheusExecutor) Execute(ctx context.Context, queries tsdb.QuerySlice, queryContext *tsdb.QueryContext) *tsdb.BatchResult {
|
||||
@@ -92,7 +95,7 @@ func (e *PrometheusExecutor) Execute(ctx context.Context, queries tsdb.QuerySlic
|
||||
return result.WithError(err)
|
||||
}
|
||||
|
||||
timeRange := prometheus.Range{
|
||||
timeRange := apiv1.Range{
|
||||
Start: query.Start,
|
||||
End: query.End,
|
||||
Step: query.Step,
|
||||
@@ -112,7 +115,7 @@ func (e *PrometheusExecutor) Execute(ctx context.Context, queries tsdb.QuerySlic
|
||||
return result
|
||||
}
|
||||
|
||||
func formatLegend(metric pmodel.Metric, query *PrometheusQuery) string {
|
||||
func formatLegend(metric model.Metric, query *PrometheusQuery) string {
|
||||
if query.LegendFormat == "" {
|
||||
return metric.String()
|
||||
}
|
||||
@@ -121,7 +124,7 @@ func formatLegend(metric pmodel.Metric, query *PrometheusQuery) string {
|
||||
labelName := strings.Replace(string(in), "{{", "", 1)
|
||||
labelName = strings.Replace(labelName, "}}", "", 1)
|
||||
labelName = strings.TrimSpace(labelName)
|
||||
if val, exists := metric[pmodel.LabelName(labelName)]; exists {
|
||||
if val, exists := metric[model.LabelName(labelName)]; exists {
|
||||
return []byte(val)
|
||||
}
|
||||
|
||||
@@ -165,11 +168,11 @@ func parseQuery(queries tsdb.QuerySlice, queryContext *tsdb.QueryContext) (*Prom
|
||||
}, nil
|
||||
}
|
||||
|
||||
func parseResponse(value pmodel.Value, query *PrometheusQuery) (map[string]*tsdb.QueryResult, error) {
|
||||
func parseResponse(value model.Value, query *PrometheusQuery) (map[string]*tsdb.QueryResult, error) {
|
||||
queryResults := make(map[string]*tsdb.QueryResult)
|
||||
queryRes := tsdb.NewQueryResult()
|
||||
|
||||
data, ok := value.(pmodel.Matrix)
|
||||
data, ok := value.(model.Matrix)
|
||||
if !ok {
|
||||
return queryResults, fmt.Errorf("Unsupported result format: %s", value.Type().String())
|
||||
}
|
||||
|
||||
@@ -40,11 +40,11 @@ const DEFAULT_MAX_LINES = 10;
|
||||
const DEFAULT_TAB_SIZE = 2;
|
||||
const DEFAULT_BEHAVIOURS = true;
|
||||
|
||||
const GRAFANA_MODULES = ['mode-prometheus', 'snippets-prometheus', 'theme-grafana-dark'];
|
||||
const GRAFANA_MODULES = ['theme-grafana-dark'];
|
||||
const GRAFANA_MODULE_BASE = "public/app/core/components/code_editor/";
|
||||
|
||||
// Trick for loading additional modules
|
||||
function setModuleUrl(moduleType, name) {
|
||||
function setModuleUrl(moduleType, name, pluginBaseUrl = null) {
|
||||
let baseUrl = ACE_SRC_BASE;
|
||||
let aceModeName = `ace/${moduleType}/${name}`;
|
||||
let moduleName = `${moduleType}-${name}`;
|
||||
@@ -54,6 +54,10 @@ function setModuleUrl(moduleType, name) {
|
||||
baseUrl = GRAFANA_MODULE_BASE;
|
||||
}
|
||||
|
||||
if (pluginBaseUrl) {
|
||||
baseUrl = pluginBaseUrl + '/';
|
||||
}
|
||||
|
||||
if (moduleType === 'snippets') {
|
||||
componentName = `${moduleType}/${name}.js`;
|
||||
}
|
||||
@@ -159,8 +163,8 @@ function link(scope, elem, attrs) {
|
||||
|
||||
function setLangMode(lang) {
|
||||
let aceModeName = `ace/mode/${lang}`;
|
||||
setModuleUrl("mode", lang);
|
||||
setModuleUrl("snippets", lang);
|
||||
setModuleUrl("mode", lang, scope.datasource.meta.baseUrl || null);
|
||||
setModuleUrl("snippets", lang, scope.datasource.meta.baseUrl || null);
|
||||
editorSession.setMode(aceModeName);
|
||||
|
||||
ace.config.loadModule("ace/ext/language_tools", (language_tools) => {
|
||||
@@ -210,6 +214,7 @@ export function codeEditorDirective() {
|
||||
template: editorTemplate,
|
||||
scope: {
|
||||
content: "=",
|
||||
datasource: "=",
|
||||
codeEditorFocus: "<",
|
||||
onChange: "&",
|
||||
getCompleter: "&"
|
||||
|
||||
@@ -395,6 +395,7 @@ function($, _) {
|
||||
kbn.valueFormats.currencyEUR = kbn.formatBuilders.currency('€');
|
||||
kbn.valueFormats.currencyJPY = kbn.formatBuilders.currency('¥');
|
||||
kbn.valueFormats.currencyRUB = kbn.formatBuilders.currency('₽');
|
||||
kbn.valueFormats.currencyUAH = kbn.formatBuilders.currency('₴');
|
||||
|
||||
// Data (Binary)
|
||||
kbn.valueFormats.bits = kbn.formatBuilders.binarySIPrefix('b');
|
||||
@@ -718,6 +719,7 @@ function($, _) {
|
||||
{text: 'Euro (€)', value: 'currencyEUR'},
|
||||
{text: 'Yen (¥)', value: 'currencyJPY'},
|
||||
{text: 'Rubles (₽)', value: 'currencyRUB'},
|
||||
{text: 'Hryvnias (₴)', value: 'currencyUAH'},
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -116,16 +116,14 @@ class TimeSrv {
|
||||
|
||||
setAutoRefresh(interval) {
|
||||
this.dashboard.refresh = interval;
|
||||
this.cancelNextRefresh();
|
||||
if (interval) {
|
||||
var intervalMs = kbn.interval_to_ms(interval);
|
||||
|
||||
this.$timeout(() => {
|
||||
this.refreshTimer = this.timer.register(this.$timeout(() => {
|
||||
this.startNextRefreshTimer(intervalMs);
|
||||
this.refreshDashboard();
|
||||
}, intervalMs);
|
||||
|
||||
} else {
|
||||
this.cancelNextRefresh();
|
||||
}, intervalMs));
|
||||
}
|
||||
|
||||
// update url
|
||||
|
||||
@@ -9,7 +9,7 @@ export class MetricsTabCtrl {
|
||||
panel: any;
|
||||
panelCtrl: any;
|
||||
datasources: any[];
|
||||
current: any;
|
||||
datasourceInstance: any;
|
||||
nextRefId: string;
|
||||
dashboard: DashboardModel;
|
||||
panelDsValue: any;
|
||||
@@ -29,23 +29,26 @@ export class MetricsTabCtrl {
|
||||
this.panel = this.panelCtrl.panel;
|
||||
this.dashboard = this.panelCtrl.dashboard;
|
||||
this.datasources = datasourceSrv.getMetricSources();
|
||||
this.panelDsValue = this.panelCtrl.panel.datasource || null;
|
||||
this.panelDsValue = this.panelCtrl.panel.datasource;
|
||||
|
||||
for (let ds of this.datasources) {
|
||||
if (ds.value === this.panelDsValue) {
|
||||
this.current = ds;
|
||||
this.datasourceInstance = ds;
|
||||
}
|
||||
}
|
||||
|
||||
this.addQueryDropdown = {text: 'Add Query', value: null, fake: true};
|
||||
|
||||
// update next ref id
|
||||
this.panelCtrl.nextRefId = this.dashboard.getNextQueryLetter(this.panel);
|
||||
this.updateDatasourceOptions();
|
||||
}
|
||||
|
||||
updateDatasourceOptions() {
|
||||
this.hasQueryHelp = this.current.meta.hasQueryHelp;
|
||||
this.queryOptions = this.current.meta.queryOptions;
|
||||
if (this.datasourceInstance) {
|
||||
this.hasQueryHelp = this.datasourceInstance.meta.hasQueryHelp;
|
||||
this.queryOptions = this.datasourceInstance.meta.queryOptions;
|
||||
}
|
||||
}
|
||||
|
||||
getOptions(includeBuiltin) {
|
||||
@@ -61,7 +64,7 @@ export class MetricsTabCtrl {
|
||||
return;
|
||||
}
|
||||
|
||||
this.current = option.datasource;
|
||||
this.datasourceInstance = option.datasource;
|
||||
this.panelCtrl.setDatasource(option.datasource);
|
||||
this.updateDatasourceOptions();
|
||||
}
|
||||
@@ -85,7 +88,7 @@ export class MetricsTabCtrl {
|
||||
this.queryTroubleshooterOpen = false;
|
||||
this.helpOpen = !this.helpOpen;
|
||||
|
||||
this.backendSrv.get(`/api/plugins/${this.current.meta.id}/markdown/query_help`).then(res => {
|
||||
this.backendSrv.get(`/api/plugins/${this.datasourceInstance.meta.id}/markdown/query_help`).then(res => {
|
||||
var md = new Remarkable();
|
||||
this.helpHtml = this.$sce.trustAsHtml(md.render(res));
|
||||
});
|
||||
|
||||
@@ -73,7 +73,7 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="query-editor-rows gf-form-group">
|
||||
<div class="query-editor-rows gf-form-group" ng-if="ctrl.datasourceInstance">
|
||||
<div ng-repeat="target in ctrl.panel.targets" ng-class="{'gf-form-disabled': target.hide}">
|
||||
<rebuild-on-change property="ctrl.panel.datasource || target.datasource" show-null="true">
|
||||
<plugin-component type="query-ctrl">
|
||||
@@ -89,11 +89,11 @@
|
||||
</span>
|
||||
<span class="gf-form-query-letter-cell-letter">{{ctrl.panelCtrl.nextRefId}}</span>
|
||||
</label>
|
||||
<button class="btn btn-secondary gf-form-btn" ng-click="ctrl.addQuery()" ng-hide="ctrl.current.meta.mixed">
|
||||
<button class="btn btn-secondary gf-form-btn" ng-click="ctrl.addQuery()" ng-hide="ctrl.datasourceInstance.meta.mixed">
|
||||
Add Query
|
||||
</button>
|
||||
|
||||
<div class="dropdown" ng-if="ctrl.current.meta.mixed">
|
||||
<div class="dropdown" ng-if="ctrl.datasourceInstance.meta.mixed">
|
||||
<gf-form-dropdown model="ctrl.addQueryDropdown" get-options="ctrl.getOptions(false)" on-change="ctrl.addMixedQuery($option)">
|
||||
</gf-form-dropdown>
|
||||
</div>
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<query-editor-row query-ctrl="ctrl" can-collapse="true" has-text-edit-mode="false">
|
||||
<div class="gf-form-inline">
|
||||
<div class="gf-form gf-form--grow">
|
||||
<code-editor content="ctrl.target.expr" on-change="ctrl.refreshMetricData()"
|
||||
<code-editor content="ctrl.target.expr" datasource="ctrl.datasource" on-change="ctrl.refreshMetricData()"
|
||||
get-completer="ctrl.getCompleter()" data-mode="prometheus" code-editor-focus="ctrl.isLastQuery">
|
||||
</code-editor>
|
||||
</div>
|
||||
|
||||
@@ -48,26 +48,14 @@
|
||||
<div class="gf-form">
|
||||
<label class="gf-form-label width-6">Mode</label>
|
||||
<div class="gf-form-select-wrapper max-width-15">
|
||||
<select class="gf-form-input" ng-model="ctrl.panel.xaxis.mode" ng-options="v as k for (k, v) in ctrl.xAxisModes" ng-change="ctrl.xAxisOptionChanged()"> </select>
|
||||
<select class="gf-form-input" ng-model="ctrl.panel.xaxis.mode" ng-options="v as k for (k, v) in ctrl.xAxisModes" ng-change="ctrl.xAxisModeChanged()"> </select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Table mode -->
|
||||
<div class="gf-form" ng-if="ctrl.panel.xaxis.mode === 'field'">
|
||||
<label class="gf-form-label width-6">Name</label>
|
||||
<metric-segment-model property="ctrl.panel.xaxis.name" get-options="ctrl.getDataFieldNames(false)" on-change="ctrl.xAxisOptionChanged()" custom="false" css-class="width-10" select-mode="true"></metric-segment-model>
|
||||
</div>
|
||||
|
||||
<!-- Series mode -->
|
||||
<div class="gf-form" ng-if="ctrl.panel.xaxis.mode === 'field'">
|
||||
<label class="gf-form-label width-6">Value</label>
|
||||
<metric-segment-model property="ctrl.panel.xaxis.values[0]" get-options="ctrl.getDataFieldNames(true)" on-change="ctrl.xAxisOptionChanged()" custom="false" css-class="width-10" select-mode="true"></metric-segment-model>
|
||||
</div>
|
||||
|
||||
<!-- Series mode -->
|
||||
<div class="gf-form" ng-if="ctrl.panel.xaxis.mode === 'series'">
|
||||
<label class="gf-form-label width-6">Value</label>
|
||||
<metric-segment-model property="ctrl.panel.xaxis.values[0]" options="ctrl.xAxisStatOptions" on-change="ctrl.xAxisOptionChanged()" custom="false" css-class="width-10" select-mode="true"></metric-segment-model>
|
||||
<metric-segment-model property="ctrl.panel.xaxis.values[0]" options="ctrl.xAxisStatOptions" on-change="ctrl.xAxisValueChanged()" custom="false" css-class="width-10" select-mode="true"></metric-segment-model>
|
||||
</div>
|
||||
|
||||
<!-- Histogram mode -->
|
||||
|
||||
@@ -59,10 +59,12 @@ export class AxesEditorCtrl {
|
||||
this.panelCtrl.render();
|
||||
}
|
||||
|
||||
xAxisOptionChanged() {
|
||||
if (!this.panel.xaxis.values || !this.panel.xaxis.values[0]){
|
||||
this.panelCtrl.processor.setPanelDefaultsForNewXAxisMode();
|
||||
}
|
||||
xAxisModeChanged() {
|
||||
this.panelCtrl.processor.setPanelDefaultsForNewXAxisMode();
|
||||
this.panelCtrl.onDataReceived(this.panelCtrl.dataList);
|
||||
}
|
||||
|
||||
xAxisValueChanged() {
|
||||
this.panelCtrl.onDataReceived(this.panelCtrl.dataList);
|
||||
}
|
||||
|
||||
|
||||
@@ -296,7 +296,7 @@ a.external-link {
|
||||
max-width: 100%;
|
||||
}
|
||||
|
||||
ul {
|
||||
ul, ol {
|
||||
padding-left: $spacer*1.5;
|
||||
margin-bottom: $spacer;
|
||||
}
|
||||
@@ -328,7 +328,7 @@ a.external-link {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
ul:last-child {
|
||||
ul:last-child, ol:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,8 +44,8 @@
|
||||
min-width: 400px;
|
||||
max-width: 600px;
|
||||
position: fixed;
|
||||
right: 20px;
|
||||
top: 56px;
|
||||
right: 10px;
|
||||
top: 60px;
|
||||
}
|
||||
|
||||
.alert-close {
|
||||
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
Copyright (C) 2013 Blake Mizerany
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of this software and associated documentation files (the
|
||||
"Software"), to deal in the Software without restriction, including
|
||||
without limitation the rights to use, copy, modify, merge, publish,
|
||||
distribute, sublicense, and/or sell copies of the Software, and to
|
||||
permit persons to whom the Software is furnished to do so, subject to
|
||||
the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be
|
||||
included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
||||
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
+2388
File diff suppressed because it is too large
Load Diff
+292
@@ -0,0 +1,292 @@
|
||||
// Package quantile computes approximate quantiles over an unbounded data
|
||||
// stream within low memory and CPU bounds.
|
||||
//
|
||||
// A small amount of accuracy is traded to achieve the above properties.
|
||||
//
|
||||
// Multiple streams can be merged before calling Query to generate a single set
|
||||
// of results. This is meaningful when the streams represent the same type of
|
||||
// data. See Merge and Samples.
|
||||
//
|
||||
// For more detailed information about the algorithm used, see:
|
||||
//
|
||||
// Effective Computation of Biased Quantiles over Data Streams
|
||||
//
|
||||
// http://www.cs.rutgers.edu/~muthu/bquant.pdf
|
||||
package quantile
|
||||
|
||||
import (
|
||||
"math"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Sample holds an observed value and meta information for compression. JSON
|
||||
// tags have been added for convenience.
|
||||
type Sample struct {
|
||||
Value float64 `json:",string"`
|
||||
Width float64 `json:",string"`
|
||||
Delta float64 `json:",string"`
|
||||
}
|
||||
|
||||
// Samples represents a slice of samples. It implements sort.Interface.
|
||||
type Samples []Sample
|
||||
|
||||
func (a Samples) Len() int { return len(a) }
|
||||
func (a Samples) Less(i, j int) bool { return a[i].Value < a[j].Value }
|
||||
func (a Samples) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
|
||||
|
||||
type invariant func(s *stream, r float64) float64
|
||||
|
||||
// NewLowBiased returns an initialized Stream for low-biased quantiles
|
||||
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||
// error guarantees can still be given even for the lower ranks of the data
|
||||
// distribution.
|
||||
//
|
||||
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||
// returned by a query is guaranteed to be within (1±Epsilon)*Quantile.
|
||||
//
|
||||
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||
// properties.
|
||||
func NewLowBiased(epsilon float64) *Stream {
|
||||
ƒ := func(s *stream, r float64) float64 {
|
||||
return 2 * epsilon * r
|
||||
}
|
||||
return newStream(ƒ)
|
||||
}
|
||||
|
||||
// NewHighBiased returns an initialized Stream for high-biased quantiles
|
||||
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||
// error guarantees can still be given even for the higher ranks of the data
|
||||
// distribution.
|
||||
//
|
||||
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||
// returned by a query is guaranteed to be within 1-(1±Epsilon)*(1-Quantile).
|
||||
//
|
||||
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||
// properties.
|
||||
func NewHighBiased(epsilon float64) *Stream {
|
||||
ƒ := func(s *stream, r float64) float64 {
|
||||
return 2 * epsilon * (s.n - r)
|
||||
}
|
||||
return newStream(ƒ)
|
||||
}
|
||||
|
||||
// NewTargeted returns an initialized Stream concerned with a particular set of
|
||||
// quantile values that are supplied a priori. Knowing these a priori reduces
|
||||
// space and computation time. The targets map maps the desired quantiles to
|
||||
// their absolute errors, i.e. the true quantile of a value returned by a query
|
||||
// is guaranteed to be within (Quantile±Epsilon).
|
||||
//
|
||||
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error properties.
|
||||
func NewTargeted(targets map[float64]float64) *Stream {
|
||||
ƒ := func(s *stream, r float64) float64 {
|
||||
var m = math.MaxFloat64
|
||||
var f float64
|
||||
for quantile, epsilon := range targets {
|
||||
if quantile*s.n <= r {
|
||||
f = (2 * epsilon * r) / quantile
|
||||
} else {
|
||||
f = (2 * epsilon * (s.n - r)) / (1 - quantile)
|
||||
}
|
||||
if f < m {
|
||||
m = f
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
return newStream(ƒ)
|
||||
}
|
||||
|
||||
// Stream computes quantiles for a stream of float64s. It is not thread-safe by
|
||||
// design. Take care when using across multiple goroutines.
|
||||
type Stream struct {
|
||||
*stream
|
||||
b Samples
|
||||
sorted bool
|
||||
}
|
||||
|
||||
func newStream(ƒ invariant) *Stream {
|
||||
x := &stream{ƒ: ƒ}
|
||||
return &Stream{x, make(Samples, 0, 500), true}
|
||||
}
|
||||
|
||||
// Insert inserts v into the stream.
|
||||
func (s *Stream) Insert(v float64) {
|
||||
s.insert(Sample{Value: v, Width: 1})
|
||||
}
|
||||
|
||||
func (s *Stream) insert(sample Sample) {
|
||||
s.b = append(s.b, sample)
|
||||
s.sorted = false
|
||||
if len(s.b) == cap(s.b) {
|
||||
s.flush()
|
||||
}
|
||||
}
|
||||
|
||||
// Query returns the computed qth percentiles value. If s was created with
|
||||
// NewTargeted, and q is not in the set of quantiles provided a priori, Query
|
||||
// will return an unspecified result.
|
||||
func (s *Stream) Query(q float64) float64 {
|
||||
if !s.flushed() {
|
||||
// Fast path when there hasn't been enough data for a flush;
|
||||
// this also yields better accuracy for small sets of data.
|
||||
l := len(s.b)
|
||||
if l == 0 {
|
||||
return 0
|
||||
}
|
||||
i := int(math.Ceil(float64(l) * q))
|
||||
if i > 0 {
|
||||
i -= 1
|
||||
}
|
||||
s.maybeSort()
|
||||
return s.b[i].Value
|
||||
}
|
||||
s.flush()
|
||||
return s.stream.query(q)
|
||||
}
|
||||
|
||||
// Merge merges samples into the underlying streams samples. This is handy when
|
||||
// merging multiple streams from separate threads, database shards, etc.
|
||||
//
|
||||
// ATTENTION: This method is broken and does not yield correct results. The
|
||||
// underlying algorithm is not capable of merging streams correctly.
|
||||
func (s *Stream) Merge(samples Samples) {
|
||||
sort.Sort(samples)
|
||||
s.stream.merge(samples)
|
||||
}
|
||||
|
||||
// Reset reinitializes and clears the list reusing the samples buffer memory.
|
||||
func (s *Stream) Reset() {
|
||||
s.stream.reset()
|
||||
s.b = s.b[:0]
|
||||
}
|
||||
|
||||
// Samples returns stream samples held by s.
|
||||
func (s *Stream) Samples() Samples {
|
||||
if !s.flushed() {
|
||||
return s.b
|
||||
}
|
||||
s.flush()
|
||||
return s.stream.samples()
|
||||
}
|
||||
|
||||
// Count returns the total number of samples observed in the stream
|
||||
// since initialization.
|
||||
func (s *Stream) Count() int {
|
||||
return len(s.b) + s.stream.count()
|
||||
}
|
||||
|
||||
func (s *Stream) flush() {
|
||||
s.maybeSort()
|
||||
s.stream.merge(s.b)
|
||||
s.b = s.b[:0]
|
||||
}
|
||||
|
||||
func (s *Stream) maybeSort() {
|
||||
if !s.sorted {
|
||||
s.sorted = true
|
||||
sort.Sort(s.b)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Stream) flushed() bool {
|
||||
return len(s.stream.l) > 0
|
||||
}
|
||||
|
||||
type stream struct {
|
||||
n float64
|
||||
l []Sample
|
||||
ƒ invariant
|
||||
}
|
||||
|
||||
func (s *stream) reset() {
|
||||
s.l = s.l[:0]
|
||||
s.n = 0
|
||||
}
|
||||
|
||||
func (s *stream) insert(v float64) {
|
||||
s.merge(Samples{{v, 1, 0}})
|
||||
}
|
||||
|
||||
func (s *stream) merge(samples Samples) {
|
||||
// TODO(beorn7): This tries to merge not only individual samples, but
|
||||
// whole summaries. The paper doesn't mention merging summaries at
|
||||
// all. Unittests show that the merging is inaccurate. Find out how to
|
||||
// do merges properly.
|
||||
var r float64
|
||||
i := 0
|
||||
for _, sample := range samples {
|
||||
for ; i < len(s.l); i++ {
|
||||
c := s.l[i]
|
||||
if c.Value > sample.Value {
|
||||
// Insert at position i.
|
||||
s.l = append(s.l, Sample{})
|
||||
copy(s.l[i+1:], s.l[i:])
|
||||
s.l[i] = Sample{
|
||||
sample.Value,
|
||||
sample.Width,
|
||||
math.Max(sample.Delta, math.Floor(s.ƒ(s, r))-1),
|
||||
// TODO(beorn7): How to calculate delta correctly?
|
||||
}
|
||||
i++
|
||||
goto inserted
|
||||
}
|
||||
r += c.Width
|
||||
}
|
||||
s.l = append(s.l, Sample{sample.Value, sample.Width, 0})
|
||||
i++
|
||||
inserted:
|
||||
s.n += sample.Width
|
||||
r += sample.Width
|
||||
}
|
||||
s.compress()
|
||||
}
|
||||
|
||||
func (s *stream) count() int {
|
||||
return int(s.n)
|
||||
}
|
||||
|
||||
func (s *stream) query(q float64) float64 {
|
||||
t := math.Ceil(q * s.n)
|
||||
t += math.Ceil(s.ƒ(s, t) / 2)
|
||||
p := s.l[0]
|
||||
var r float64
|
||||
for _, c := range s.l[1:] {
|
||||
r += p.Width
|
||||
if r+c.Width+c.Delta > t {
|
||||
return p.Value
|
||||
}
|
||||
p = c
|
||||
}
|
||||
return p.Value
|
||||
}
|
||||
|
||||
func (s *stream) compress() {
|
||||
if len(s.l) < 2 {
|
||||
return
|
||||
}
|
||||
x := s.l[len(s.l)-1]
|
||||
xi := len(s.l) - 1
|
||||
r := s.n - 1 - x.Width
|
||||
|
||||
for i := len(s.l) - 2; i >= 0; i-- {
|
||||
c := s.l[i]
|
||||
if c.Width+x.Width+x.Delta <= s.ƒ(s, r) {
|
||||
x.Width += c.Width
|
||||
s.l[xi] = x
|
||||
// Remove element at i.
|
||||
copy(s.l[i:], s.l[i+1:])
|
||||
s.l = s.l[:len(s.l)-1]
|
||||
xi -= 1
|
||||
} else {
|
||||
x = c
|
||||
xi = i
|
||||
}
|
||||
r -= c.Width
|
||||
}
|
||||
}
|
||||
|
||||
func (s *stream) samples() Samples {
|
||||
samples := make(Samples, len(s.l))
|
||||
copy(samples, s.l)
|
||||
return samples
|
||||
}
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
Go support for Protocol Buffers - Google's data interchange format
|
||||
|
||||
Copyright 2010 The Go Authors. All rights reserved.
|
||||
https://github.com/golang/protobuf
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
* Redistributions in binary form must reproduce the above
|
||||
copyright notice, this list of conditions and the following disclaimer
|
||||
in the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
* Neither the name of Google Inc. nor the names of its
|
||||
contributors may be used to endorse or promote products derived from
|
||||
this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Go support for Protocol Buffers - Google's data interchange format
|
||||
#
|
||||
# Copyright 2010 The Go Authors. All rights reserved.
|
||||
# https://github.com/golang/protobuf
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions are
|
||||
# met:
|
||||
#
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above
|
||||
# copyright notice, this list of conditions and the following disclaimer
|
||||
# in the documentation and/or other materials provided with the
|
||||
# distribution.
|
||||
# * Neither the name of Google Inc. nor the names of its
|
||||
# contributors may be used to endorse or promote products derived from
|
||||
# this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
install:
|
||||
go install
|
||||
|
||||
test: install generate-test-pbs
|
||||
go test
|
||||
|
||||
|
||||
generate-test-pbs:
|
||||
make install
|
||||
make -C testdata
|
||||
protoc --go_out=Mtestdata/test.proto=github.com/golang/protobuf/proto/testdata,Mgoogle/protobuf/any.proto=github.com/golang/protobuf/ptypes/any:. proto3_proto/proto3.proto
|
||||
make
|
||||
+229
@@ -0,0 +1,229 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2011 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
// Protocol buffer deep copy and merge.
|
||||
// TODO: RawMessage.
|
||||
|
||||
package proto
|
||||
|
||||
import (
|
||||
"log"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Clone returns a deep copy of a protocol buffer.
|
||||
func Clone(pb Message) Message {
|
||||
in := reflect.ValueOf(pb)
|
||||
if in.IsNil() {
|
||||
return pb
|
||||
}
|
||||
|
||||
out := reflect.New(in.Type().Elem())
|
||||
// out is empty so a merge is a deep copy.
|
||||
mergeStruct(out.Elem(), in.Elem())
|
||||
return out.Interface().(Message)
|
||||
}
|
||||
|
||||
// Merge merges src into dst.
|
||||
// Required and optional fields that are set in src will be set to that value in dst.
|
||||
// Elements of repeated fields will be appended.
|
||||
// Merge panics if src and dst are not the same type, or if dst is nil.
|
||||
func Merge(dst, src Message) {
|
||||
in := reflect.ValueOf(src)
|
||||
out := reflect.ValueOf(dst)
|
||||
if out.IsNil() {
|
||||
panic("proto: nil destination")
|
||||
}
|
||||
if in.Type() != out.Type() {
|
||||
// Explicit test prior to mergeStruct so that mistyped nils will fail
|
||||
panic("proto: type mismatch")
|
||||
}
|
||||
if in.IsNil() {
|
||||
// Merging nil into non-nil is a quiet no-op
|
||||
return
|
||||
}
|
||||
mergeStruct(out.Elem(), in.Elem())
|
||||
}
|
||||
|
||||
func mergeStruct(out, in reflect.Value) {
|
||||
sprop := GetProperties(in.Type())
|
||||
for i := 0; i < in.NumField(); i++ {
|
||||
f := in.Type().Field(i)
|
||||
if strings.HasPrefix(f.Name, "XXX_") {
|
||||
continue
|
||||
}
|
||||
mergeAny(out.Field(i), in.Field(i), false, sprop.Prop[i])
|
||||
}
|
||||
|
||||
if emIn, ok := extendable(in.Addr().Interface()); ok {
|
||||
emOut, _ := extendable(out.Addr().Interface())
|
||||
mIn, muIn := emIn.extensionsRead()
|
||||
if mIn != nil {
|
||||
mOut := emOut.extensionsWrite()
|
||||
muIn.Lock()
|
||||
mergeExtension(mOut, mIn)
|
||||
muIn.Unlock()
|
||||
}
|
||||
}
|
||||
|
||||
uf := in.FieldByName("XXX_unrecognized")
|
||||
if !uf.IsValid() {
|
||||
return
|
||||
}
|
||||
uin := uf.Bytes()
|
||||
if len(uin) > 0 {
|
||||
out.FieldByName("XXX_unrecognized").SetBytes(append([]byte(nil), uin...))
|
||||
}
|
||||
}
|
||||
|
||||
// mergeAny performs a merge between two values of the same type.
|
||||
// viaPtr indicates whether the values were indirected through a pointer (implying proto2).
|
||||
// prop is set if this is a struct field (it may be nil).
|
||||
func mergeAny(out, in reflect.Value, viaPtr bool, prop *Properties) {
|
||||
if in.Type() == protoMessageType {
|
||||
if !in.IsNil() {
|
||||
if out.IsNil() {
|
||||
out.Set(reflect.ValueOf(Clone(in.Interface().(Message))))
|
||||
} else {
|
||||
Merge(out.Interface().(Message), in.Interface().(Message))
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
switch in.Kind() {
|
||||
case reflect.Bool, reflect.Float32, reflect.Float64, reflect.Int32, reflect.Int64,
|
||||
reflect.String, reflect.Uint32, reflect.Uint64:
|
||||
if !viaPtr && isProto3Zero(in) {
|
||||
return
|
||||
}
|
||||
out.Set(in)
|
||||
case reflect.Interface:
|
||||
// Probably a oneof field; copy non-nil values.
|
||||
if in.IsNil() {
|
||||
return
|
||||
}
|
||||
// Allocate destination if it is not set, or set to a different type.
|
||||
// Otherwise we will merge as normal.
|
||||
if out.IsNil() || out.Elem().Type() != in.Elem().Type() {
|
||||
out.Set(reflect.New(in.Elem().Elem().Type())) // interface -> *T -> T -> new(T)
|
||||
}
|
||||
mergeAny(out.Elem(), in.Elem(), false, nil)
|
||||
case reflect.Map:
|
||||
if in.Len() == 0 {
|
||||
return
|
||||
}
|
||||
if out.IsNil() {
|
||||
out.Set(reflect.MakeMap(in.Type()))
|
||||
}
|
||||
// For maps with value types of *T or []byte we need to deep copy each value.
|
||||
elemKind := in.Type().Elem().Kind()
|
||||
for _, key := range in.MapKeys() {
|
||||
var val reflect.Value
|
||||
switch elemKind {
|
||||
case reflect.Ptr:
|
||||
val = reflect.New(in.Type().Elem().Elem())
|
||||
mergeAny(val, in.MapIndex(key), false, nil)
|
||||
case reflect.Slice:
|
||||
val = in.MapIndex(key)
|
||||
val = reflect.ValueOf(append([]byte{}, val.Bytes()...))
|
||||
default:
|
||||
val = in.MapIndex(key)
|
||||
}
|
||||
out.SetMapIndex(key, val)
|
||||
}
|
||||
case reflect.Ptr:
|
||||
if in.IsNil() {
|
||||
return
|
||||
}
|
||||
if out.IsNil() {
|
||||
out.Set(reflect.New(in.Elem().Type()))
|
||||
}
|
||||
mergeAny(out.Elem(), in.Elem(), true, nil)
|
||||
case reflect.Slice:
|
||||
if in.IsNil() {
|
||||
return
|
||||
}
|
||||
if in.Type().Elem().Kind() == reflect.Uint8 {
|
||||
// []byte is a scalar bytes field, not a repeated field.
|
||||
|
||||
// Edge case: if this is in a proto3 message, a zero length
|
||||
// bytes field is considered the zero value, and should not
|
||||
// be merged.
|
||||
if prop != nil && prop.proto3 && in.Len() == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// Make a deep copy.
|
||||
// Append to []byte{} instead of []byte(nil) so that we never end up
|
||||
// with a nil result.
|
||||
out.SetBytes(append([]byte{}, in.Bytes()...))
|
||||
return
|
||||
}
|
||||
n := in.Len()
|
||||
if out.IsNil() {
|
||||
out.Set(reflect.MakeSlice(in.Type(), 0, n))
|
||||
}
|
||||
switch in.Type().Elem().Kind() {
|
||||
case reflect.Bool, reflect.Float32, reflect.Float64, reflect.Int32, reflect.Int64,
|
||||
reflect.String, reflect.Uint32, reflect.Uint64:
|
||||
out.Set(reflect.AppendSlice(out, in))
|
||||
default:
|
||||
for i := 0; i < n; i++ {
|
||||
x := reflect.Indirect(reflect.New(in.Type().Elem()))
|
||||
mergeAny(x, in.Index(i), false, nil)
|
||||
out.Set(reflect.Append(out, x))
|
||||
}
|
||||
}
|
||||
case reflect.Struct:
|
||||
mergeStruct(out, in)
|
||||
default:
|
||||
// unknown type, so not a protocol buffer
|
||||
log.Printf("proto: don't know how to copy %v", in)
|
||||
}
|
||||
}
|
||||
|
||||
func mergeExtension(out, in map[int32]Extension) {
|
||||
for extNum, eIn := range in {
|
||||
eOut := Extension{desc: eIn.desc}
|
||||
if eIn.value != nil {
|
||||
v := reflect.New(reflect.TypeOf(eIn.value)).Elem()
|
||||
mergeAny(v, reflect.ValueOf(eIn.value), false, nil)
|
||||
eOut.value = v.Interface()
|
||||
}
|
||||
if eIn.enc != nil {
|
||||
eOut.enc = make([]byte, len(eIn.enc))
|
||||
copy(eOut.enc, eIn.enc)
|
||||
}
|
||||
|
||||
out[extNum] = eOut
|
||||
}
|
||||
}
|
||||
+970
@@ -0,0 +1,970 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
package proto
|
||||
|
||||
/*
|
||||
* Routines for decoding protocol buffer data to construct in-memory representations.
|
||||
*/
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"reflect"
|
||||
)
|
||||
|
||||
// errOverflow is returned when an integer is too large to be represented.
|
||||
var errOverflow = errors.New("proto: integer overflow")
|
||||
|
||||
// ErrInternalBadWireType is returned by generated code when an incorrect
|
||||
// wire type is encountered. It does not get returned to user code.
|
||||
var ErrInternalBadWireType = errors.New("proto: internal error: bad wiretype for oneof")
|
||||
|
||||
// The fundamental decoders that interpret bytes on the wire.
|
||||
// Those that take integer types all return uint64 and are
|
||||
// therefore of type valueDecoder.
|
||||
|
||||
// DecodeVarint reads a varint-encoded integer from the slice.
|
||||
// It returns the integer and the number of bytes consumed, or
|
||||
// zero if there is not enough.
|
||||
// This is the format for the
|
||||
// int32, int64, uint32, uint64, bool, and enum
|
||||
// protocol buffer types.
|
||||
func DecodeVarint(buf []byte) (x uint64, n int) {
|
||||
for shift := uint(0); shift < 64; shift += 7 {
|
||||
if n >= len(buf) {
|
||||
return 0, 0
|
||||
}
|
||||
b := uint64(buf[n])
|
||||
n++
|
||||
x |= (b & 0x7F) << shift
|
||||
if (b & 0x80) == 0 {
|
||||
return x, n
|
||||
}
|
||||
}
|
||||
|
||||
// The number is too large to represent in a 64-bit value.
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
func (p *Buffer) decodeVarintSlow() (x uint64, err error) {
|
||||
i := p.index
|
||||
l := len(p.buf)
|
||||
|
||||
for shift := uint(0); shift < 64; shift += 7 {
|
||||
if i >= l {
|
||||
err = io.ErrUnexpectedEOF
|
||||
return
|
||||
}
|
||||
b := p.buf[i]
|
||||
i++
|
||||
x |= (uint64(b) & 0x7F) << shift
|
||||
if b < 0x80 {
|
||||
p.index = i
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// The number is too large to represent in a 64-bit value.
|
||||
err = errOverflow
|
||||
return
|
||||
}
|
||||
|
||||
// DecodeVarint reads a varint-encoded integer from the Buffer.
|
||||
// This is the format for the
|
||||
// int32, int64, uint32, uint64, bool, and enum
|
||||
// protocol buffer types.
|
||||
func (p *Buffer) DecodeVarint() (x uint64, err error) {
|
||||
i := p.index
|
||||
buf := p.buf
|
||||
|
||||
if i >= len(buf) {
|
||||
return 0, io.ErrUnexpectedEOF
|
||||
} else if buf[i] < 0x80 {
|
||||
p.index++
|
||||
return uint64(buf[i]), nil
|
||||
} else if len(buf)-i < 10 {
|
||||
return p.decodeVarintSlow()
|
||||
}
|
||||
|
||||
var b uint64
|
||||
// we already checked the first byte
|
||||
x = uint64(buf[i]) - 0x80
|
||||
i++
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 7
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 7
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 14
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 14
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 21
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 21
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 28
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 28
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 35
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 35
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 42
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 42
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 49
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 49
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 56
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
x -= 0x80 << 56
|
||||
|
||||
b = uint64(buf[i])
|
||||
i++
|
||||
x += b << 63
|
||||
if b&0x80 == 0 {
|
||||
goto done
|
||||
}
|
||||
// x -= 0x80 << 63 // Always zero.
|
||||
|
||||
return 0, errOverflow
|
||||
|
||||
done:
|
||||
p.index = i
|
||||
return x, nil
|
||||
}
|
||||
|
||||
// DecodeFixed64 reads a 64-bit integer from the Buffer.
|
||||
// This is the format for the
|
||||
// fixed64, sfixed64, and double protocol buffer types.
|
||||
func (p *Buffer) DecodeFixed64() (x uint64, err error) {
|
||||
// x, err already 0
|
||||
i := p.index + 8
|
||||
if i < 0 || i > len(p.buf) {
|
||||
err = io.ErrUnexpectedEOF
|
||||
return
|
||||
}
|
||||
p.index = i
|
||||
|
||||
x = uint64(p.buf[i-8])
|
||||
x |= uint64(p.buf[i-7]) << 8
|
||||
x |= uint64(p.buf[i-6]) << 16
|
||||
x |= uint64(p.buf[i-5]) << 24
|
||||
x |= uint64(p.buf[i-4]) << 32
|
||||
x |= uint64(p.buf[i-3]) << 40
|
||||
x |= uint64(p.buf[i-2]) << 48
|
||||
x |= uint64(p.buf[i-1]) << 56
|
||||
return
|
||||
}
|
||||
|
||||
// DecodeFixed32 reads a 32-bit integer from the Buffer.
|
||||
// This is the format for the
|
||||
// fixed32, sfixed32, and float protocol buffer types.
|
||||
func (p *Buffer) DecodeFixed32() (x uint64, err error) {
|
||||
// x, err already 0
|
||||
i := p.index + 4
|
||||
if i < 0 || i > len(p.buf) {
|
||||
err = io.ErrUnexpectedEOF
|
||||
return
|
||||
}
|
||||
p.index = i
|
||||
|
||||
x = uint64(p.buf[i-4])
|
||||
x |= uint64(p.buf[i-3]) << 8
|
||||
x |= uint64(p.buf[i-2]) << 16
|
||||
x |= uint64(p.buf[i-1]) << 24
|
||||
return
|
||||
}
|
||||
|
||||
// DecodeZigzag64 reads a zigzag-encoded 64-bit integer
|
||||
// from the Buffer.
|
||||
// This is the format used for the sint64 protocol buffer type.
|
||||
func (p *Buffer) DecodeZigzag64() (x uint64, err error) {
|
||||
x, err = p.DecodeVarint()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
x = (x >> 1) ^ uint64((int64(x&1)<<63)>>63)
|
||||
return
|
||||
}
|
||||
|
||||
// DecodeZigzag32 reads a zigzag-encoded 32-bit integer
|
||||
// from the Buffer.
|
||||
// This is the format used for the sint32 protocol buffer type.
|
||||
func (p *Buffer) DecodeZigzag32() (x uint64, err error) {
|
||||
x, err = p.DecodeVarint()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
x = uint64((uint32(x) >> 1) ^ uint32((int32(x&1)<<31)>>31))
|
||||
return
|
||||
}
|
||||
|
||||
// These are not ValueDecoders: they produce an array of bytes or a string.
|
||||
// bytes, embedded messages
|
||||
|
||||
// DecodeRawBytes reads a count-delimited byte buffer from the Buffer.
|
||||
// This is the format used for the bytes protocol buffer
|
||||
// type and for embedded messages.
|
||||
func (p *Buffer) DecodeRawBytes(alloc bool) (buf []byte, err error) {
|
||||
n, err := p.DecodeVarint()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
nb := int(n)
|
||||
if nb < 0 {
|
||||
return nil, fmt.Errorf("proto: bad byte length %d", nb)
|
||||
}
|
||||
end := p.index + nb
|
||||
if end < p.index || end > len(p.buf) {
|
||||
return nil, io.ErrUnexpectedEOF
|
||||
}
|
||||
|
||||
if !alloc {
|
||||
// todo: check if can get more uses of alloc=false
|
||||
buf = p.buf[p.index:end]
|
||||
p.index += nb
|
||||
return
|
||||
}
|
||||
|
||||
buf = make([]byte, nb)
|
||||
copy(buf, p.buf[p.index:])
|
||||
p.index += nb
|
||||
return
|
||||
}
|
||||
|
||||
// DecodeStringBytes reads an encoded string from the Buffer.
|
||||
// This is the format used for the proto2 string type.
|
||||
func (p *Buffer) DecodeStringBytes() (s string, err error) {
|
||||
buf, err := p.DecodeRawBytes(false)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
return string(buf), nil
|
||||
}
|
||||
|
||||
// Skip the next item in the buffer. Its wire type is decoded and presented as an argument.
|
||||
// If the protocol buffer has extensions, and the field matches, add it as an extension.
|
||||
// Otherwise, if the XXX_unrecognized field exists, append the skipped data there.
|
||||
func (o *Buffer) skipAndSave(t reflect.Type, tag, wire int, base structPointer, unrecField field) error {
|
||||
oi := o.index
|
||||
|
||||
err := o.skip(t, tag, wire)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !unrecField.IsValid() {
|
||||
return nil
|
||||
}
|
||||
|
||||
ptr := structPointer_Bytes(base, unrecField)
|
||||
|
||||
// Add the skipped field to struct field
|
||||
obuf := o.buf
|
||||
|
||||
o.buf = *ptr
|
||||
o.EncodeVarint(uint64(tag<<3 | wire))
|
||||
*ptr = append(o.buf, obuf[oi:o.index]...)
|
||||
|
||||
o.buf = obuf
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Skip the next item in the buffer. Its wire type is decoded and presented as an argument.
|
||||
func (o *Buffer) skip(t reflect.Type, tag, wire int) error {
|
||||
|
||||
var u uint64
|
||||
var err error
|
||||
|
||||
switch wire {
|
||||
case WireVarint:
|
||||
_, err = o.DecodeVarint()
|
||||
case WireFixed64:
|
||||
_, err = o.DecodeFixed64()
|
||||
case WireBytes:
|
||||
_, err = o.DecodeRawBytes(false)
|
||||
case WireFixed32:
|
||||
_, err = o.DecodeFixed32()
|
||||
case WireStartGroup:
|
||||
for {
|
||||
u, err = o.DecodeVarint()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
fwire := int(u & 0x7)
|
||||
if fwire == WireEndGroup {
|
||||
break
|
||||
}
|
||||
ftag := int(u >> 3)
|
||||
err = o.skip(t, ftag, fwire)
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
default:
|
||||
err = fmt.Errorf("proto: can't skip unknown wire type %d for %s", wire, t)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Unmarshaler is the interface representing objects that can
|
||||
// unmarshal themselves. The method should reset the receiver before
|
||||
// decoding starts. The argument points to data that may be
|
||||
// overwritten, so implementations should not keep references to the
|
||||
// buffer.
|
||||
type Unmarshaler interface {
|
||||
Unmarshal([]byte) error
|
||||
}
|
||||
|
||||
// Unmarshal parses the protocol buffer representation in buf and places the
|
||||
// decoded result in pb. If the struct underlying pb does not match
|
||||
// the data in buf, the results can be unpredictable.
|
||||
//
|
||||
// Unmarshal resets pb before starting to unmarshal, so any
|
||||
// existing data in pb is always removed. Use UnmarshalMerge
|
||||
// to preserve and append to existing data.
|
||||
func Unmarshal(buf []byte, pb Message) error {
|
||||
pb.Reset()
|
||||
return UnmarshalMerge(buf, pb)
|
||||
}
|
||||
|
||||
// UnmarshalMerge parses the protocol buffer representation in buf and
|
||||
// writes the decoded result to pb. If the struct underlying pb does not match
|
||||
// the data in buf, the results can be unpredictable.
|
||||
//
|
||||
// UnmarshalMerge merges into existing data in pb.
|
||||
// Most code should use Unmarshal instead.
|
||||
func UnmarshalMerge(buf []byte, pb Message) error {
|
||||
// If the object can unmarshal itself, let it.
|
||||
if u, ok := pb.(Unmarshaler); ok {
|
||||
return u.Unmarshal(buf)
|
||||
}
|
||||
return NewBuffer(buf).Unmarshal(pb)
|
||||
}
|
||||
|
||||
// DecodeMessage reads a count-delimited message from the Buffer.
|
||||
func (p *Buffer) DecodeMessage(pb Message) error {
|
||||
enc, err := p.DecodeRawBytes(false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return NewBuffer(enc).Unmarshal(pb)
|
||||
}
|
||||
|
||||
// DecodeGroup reads a tag-delimited group from the Buffer.
|
||||
func (p *Buffer) DecodeGroup(pb Message) error {
|
||||
typ, base, err := getbase(pb)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return p.unmarshalType(typ.Elem(), GetProperties(typ.Elem()), true, base)
|
||||
}
|
||||
|
||||
// Unmarshal parses the protocol buffer representation in the
|
||||
// Buffer and places the decoded result in pb. If the struct
|
||||
// underlying pb does not match the data in the buffer, the results can be
|
||||
// unpredictable.
|
||||
//
|
||||
// Unlike proto.Unmarshal, this does not reset pb before starting to unmarshal.
|
||||
func (p *Buffer) Unmarshal(pb Message) error {
|
||||
// If the object can unmarshal itself, let it.
|
||||
if u, ok := pb.(Unmarshaler); ok {
|
||||
err := u.Unmarshal(p.buf[p.index:])
|
||||
p.index = len(p.buf)
|
||||
return err
|
||||
}
|
||||
|
||||
typ, base, err := getbase(pb)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = p.unmarshalType(typ.Elem(), GetProperties(typ.Elem()), false, base)
|
||||
|
||||
if collectStats {
|
||||
stats.Decode++
|
||||
}
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// unmarshalType does the work of unmarshaling a structure.
|
||||
func (o *Buffer) unmarshalType(st reflect.Type, prop *StructProperties, is_group bool, base structPointer) error {
|
||||
var state errorState
|
||||
required, reqFields := prop.reqCount, uint64(0)
|
||||
|
||||
var err error
|
||||
for err == nil && o.index < len(o.buf) {
|
||||
oi := o.index
|
||||
var u uint64
|
||||
u, err = o.DecodeVarint()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
wire := int(u & 0x7)
|
||||
if wire == WireEndGroup {
|
||||
if is_group {
|
||||
if required > 0 {
|
||||
// Not enough information to determine the exact field.
|
||||
// (See below.)
|
||||
return &RequiredNotSetError{"{Unknown}"}
|
||||
}
|
||||
return nil // input is satisfied
|
||||
}
|
||||
return fmt.Errorf("proto: %s: wiretype end group for non-group", st)
|
||||
}
|
||||
tag := int(u >> 3)
|
||||
if tag <= 0 {
|
||||
return fmt.Errorf("proto: %s: illegal tag %d (wire type %d)", st, tag, wire)
|
||||
}
|
||||
fieldnum, ok := prop.decoderTags.get(tag)
|
||||
if !ok {
|
||||
// Maybe it's an extension?
|
||||
if prop.extendable {
|
||||
if e, _ := extendable(structPointer_Interface(base, st)); isExtensionField(e, int32(tag)) {
|
||||
if err = o.skip(st, tag, wire); err == nil {
|
||||
extmap := e.extensionsWrite()
|
||||
ext := extmap[int32(tag)] // may be missing
|
||||
ext.enc = append(ext.enc, o.buf[oi:o.index]...)
|
||||
extmap[int32(tag)] = ext
|
||||
}
|
||||
continue
|
||||
}
|
||||
}
|
||||
// Maybe it's a oneof?
|
||||
if prop.oneofUnmarshaler != nil {
|
||||
m := structPointer_Interface(base, st).(Message)
|
||||
// First return value indicates whether tag is a oneof field.
|
||||
ok, err = prop.oneofUnmarshaler(m, tag, wire, o)
|
||||
if err == ErrInternalBadWireType {
|
||||
// Map the error to something more descriptive.
|
||||
// Do the formatting here to save generated code space.
|
||||
err = fmt.Errorf("bad wiretype for oneof field in %T", m)
|
||||
}
|
||||
if ok {
|
||||
continue
|
||||
}
|
||||
}
|
||||
err = o.skipAndSave(st, tag, wire, base, prop.unrecField)
|
||||
continue
|
||||
}
|
||||
p := prop.Prop[fieldnum]
|
||||
|
||||
if p.dec == nil {
|
||||
fmt.Fprintf(os.Stderr, "proto: no protobuf decoder for %s.%s\n", st, st.Field(fieldnum).Name)
|
||||
continue
|
||||
}
|
||||
dec := p.dec
|
||||
if wire != WireStartGroup && wire != p.WireType {
|
||||
if wire == WireBytes && p.packedDec != nil {
|
||||
// a packable field
|
||||
dec = p.packedDec
|
||||
} else {
|
||||
err = fmt.Errorf("proto: bad wiretype for field %s.%s: got wiretype %d, want %d", st, st.Field(fieldnum).Name, wire, p.WireType)
|
||||
continue
|
||||
}
|
||||
}
|
||||
decErr := dec(o, p, base)
|
||||
if decErr != nil && !state.shouldContinue(decErr, p) {
|
||||
err = decErr
|
||||
}
|
||||
if err == nil && p.Required {
|
||||
// Successfully decoded a required field.
|
||||
if tag <= 64 {
|
||||
// use bitmap for fields 1-64 to catch field reuse.
|
||||
var mask uint64 = 1 << uint64(tag-1)
|
||||
if reqFields&mask == 0 {
|
||||
// new required field
|
||||
reqFields |= mask
|
||||
required--
|
||||
}
|
||||
} else {
|
||||
// This is imprecise. It can be fooled by a required field
|
||||
// with a tag > 64 that is encoded twice; that's very rare.
|
||||
// A fully correct implementation would require allocating
|
||||
// a data structure, which we would like to avoid.
|
||||
required--
|
||||
}
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
if is_group {
|
||||
return io.ErrUnexpectedEOF
|
||||
}
|
||||
if state.err != nil {
|
||||
return state.err
|
||||
}
|
||||
if required > 0 {
|
||||
// Not enough information to determine the exact field. If we use extra
|
||||
// CPU, we could determine the field only if the missing required field
|
||||
// has a tag <= 64 and we check reqFields.
|
||||
return &RequiredNotSetError{"{Unknown}"}
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Individual type decoders
|
||||
// For each,
|
||||
// u is the decoded value,
|
||||
// v is a pointer to the field (pointer) in the struct
|
||||
|
||||
// Sizes of the pools to allocate inside the Buffer.
|
||||
// The goal is modest amortization and allocation
|
||||
// on at least 16-byte boundaries.
|
||||
const (
|
||||
boolPoolSize = 16
|
||||
uint32PoolSize = 8
|
||||
uint64PoolSize = 4
|
||||
)
|
||||
|
||||
// Decode a bool.
|
||||
func (o *Buffer) dec_bool(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(o.bools) == 0 {
|
||||
o.bools = make([]bool, boolPoolSize)
|
||||
}
|
||||
o.bools[0] = u != 0
|
||||
*structPointer_Bool(base, p.field) = &o.bools[0]
|
||||
o.bools = o.bools[1:]
|
||||
return nil
|
||||
}
|
||||
|
||||
func (o *Buffer) dec_proto3_bool(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*structPointer_BoolVal(base, p.field) = u != 0
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode an int32.
|
||||
func (o *Buffer) dec_int32(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
word32_Set(structPointer_Word32(base, p.field), o, uint32(u))
|
||||
return nil
|
||||
}
|
||||
|
||||
func (o *Buffer) dec_proto3_int32(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
word32Val_Set(structPointer_Word32Val(base, p.field), uint32(u))
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode an int64.
|
||||
func (o *Buffer) dec_int64(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
word64_Set(structPointer_Word64(base, p.field), o, u)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (o *Buffer) dec_proto3_int64(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
word64Val_Set(structPointer_Word64Val(base, p.field), o, u)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a string.
|
||||
func (o *Buffer) dec_string(p *Properties, base structPointer) error {
|
||||
s, err := o.DecodeStringBytes()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*structPointer_String(base, p.field) = &s
|
||||
return nil
|
||||
}
|
||||
|
||||
func (o *Buffer) dec_proto3_string(p *Properties, base structPointer) error {
|
||||
s, err := o.DecodeStringBytes()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*structPointer_StringVal(base, p.field) = s
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of bytes ([]byte).
|
||||
func (o *Buffer) dec_slice_byte(p *Properties, base structPointer) error {
|
||||
b, err := o.DecodeRawBytes(true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*structPointer_Bytes(base, p.field) = b
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of bools ([]bool).
|
||||
func (o *Buffer) dec_slice_bool(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v := structPointer_BoolSlice(base, p.field)
|
||||
*v = append(*v, u != 0)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of bools ([]bool) in packed format.
|
||||
func (o *Buffer) dec_slice_packed_bool(p *Properties, base structPointer) error {
|
||||
v := structPointer_BoolSlice(base, p.field)
|
||||
|
||||
nn, err := o.DecodeVarint()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
nb := int(nn) // number of bytes of encoded bools
|
||||
fin := o.index + nb
|
||||
if fin < o.index {
|
||||
return errOverflow
|
||||
}
|
||||
|
||||
y := *v
|
||||
for o.index < fin {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
y = append(y, u != 0)
|
||||
}
|
||||
|
||||
*v = y
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of int32s ([]int32).
|
||||
func (o *Buffer) dec_slice_int32(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
structPointer_Word32Slice(base, p.field).Append(uint32(u))
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of int32s ([]int32) in packed format.
|
||||
func (o *Buffer) dec_slice_packed_int32(p *Properties, base structPointer) error {
|
||||
v := structPointer_Word32Slice(base, p.field)
|
||||
|
||||
nn, err := o.DecodeVarint()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
nb := int(nn) // number of bytes of encoded int32s
|
||||
|
||||
fin := o.index + nb
|
||||
if fin < o.index {
|
||||
return errOverflow
|
||||
}
|
||||
for o.index < fin {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v.Append(uint32(u))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of int64s ([]int64).
|
||||
func (o *Buffer) dec_slice_int64(p *Properties, base structPointer) error {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
structPointer_Word64Slice(base, p.field).Append(u)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of int64s ([]int64) in packed format.
|
||||
func (o *Buffer) dec_slice_packed_int64(p *Properties, base structPointer) error {
|
||||
v := structPointer_Word64Slice(base, p.field)
|
||||
|
||||
nn, err := o.DecodeVarint()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
nb := int(nn) // number of bytes of encoded int64s
|
||||
|
||||
fin := o.index + nb
|
||||
if fin < o.index {
|
||||
return errOverflow
|
||||
}
|
||||
for o.index < fin {
|
||||
u, err := p.valDec(o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v.Append(u)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of strings ([]string).
|
||||
func (o *Buffer) dec_slice_string(p *Properties, base structPointer) error {
|
||||
s, err := o.DecodeStringBytes()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v := structPointer_StringSlice(base, p.field)
|
||||
*v = append(*v, s)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a slice of slice of bytes ([][]byte).
|
||||
func (o *Buffer) dec_slice_slice_byte(p *Properties, base structPointer) error {
|
||||
b, err := o.DecodeRawBytes(true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v := structPointer_BytesSlice(base, p.field)
|
||||
*v = append(*v, b)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a map field.
|
||||
func (o *Buffer) dec_new_map(p *Properties, base structPointer) error {
|
||||
raw, err := o.DecodeRawBytes(false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
oi := o.index // index at the end of this map entry
|
||||
o.index -= len(raw) // move buffer back to start of map entry
|
||||
|
||||
mptr := structPointer_NewAt(base, p.field, p.mtype) // *map[K]V
|
||||
if mptr.Elem().IsNil() {
|
||||
mptr.Elem().Set(reflect.MakeMap(mptr.Type().Elem()))
|
||||
}
|
||||
v := mptr.Elem() // map[K]V
|
||||
|
||||
// Prepare addressable doubly-indirect placeholders for the key and value types.
|
||||
// See enc_new_map for why.
|
||||
keyptr := reflect.New(reflect.PtrTo(p.mtype.Key())).Elem() // addressable *K
|
||||
keybase := toStructPointer(keyptr.Addr()) // **K
|
||||
|
||||
var valbase structPointer
|
||||
var valptr reflect.Value
|
||||
switch p.mtype.Elem().Kind() {
|
||||
case reflect.Slice:
|
||||
// []byte
|
||||
var dummy []byte
|
||||
valptr = reflect.ValueOf(&dummy) // *[]byte
|
||||
valbase = toStructPointer(valptr) // *[]byte
|
||||
case reflect.Ptr:
|
||||
// message; valptr is **Msg; need to allocate the intermediate pointer
|
||||
valptr = reflect.New(reflect.PtrTo(p.mtype.Elem())).Elem() // addressable *V
|
||||
valptr.Set(reflect.New(valptr.Type().Elem()))
|
||||
valbase = toStructPointer(valptr)
|
||||
default:
|
||||
// everything else
|
||||
valptr = reflect.New(reflect.PtrTo(p.mtype.Elem())).Elem() // addressable *V
|
||||
valbase = toStructPointer(valptr.Addr()) // **V
|
||||
}
|
||||
|
||||
// Decode.
|
||||
// This parses a restricted wire format, namely the encoding of a message
|
||||
// with two fields. See enc_new_map for the format.
|
||||
for o.index < oi {
|
||||
// tagcode for key and value properties are always a single byte
|
||||
// because they have tags 1 and 2.
|
||||
tagcode := o.buf[o.index]
|
||||
o.index++
|
||||
switch tagcode {
|
||||
case p.mkeyprop.tagcode[0]:
|
||||
if err := p.mkeyprop.dec(o, p.mkeyprop, keybase); err != nil {
|
||||
return err
|
||||
}
|
||||
case p.mvalprop.tagcode[0]:
|
||||
if err := p.mvalprop.dec(o, p.mvalprop, valbase); err != nil {
|
||||
return err
|
||||
}
|
||||
default:
|
||||
// TODO: Should we silently skip this instead?
|
||||
return fmt.Errorf("proto: bad map data tag %d", raw[0])
|
||||
}
|
||||
}
|
||||
keyelem, valelem := keyptr.Elem(), valptr.Elem()
|
||||
if !keyelem.IsValid() {
|
||||
keyelem = reflect.Zero(p.mtype.Key())
|
||||
}
|
||||
if !valelem.IsValid() {
|
||||
valelem = reflect.Zero(p.mtype.Elem())
|
||||
}
|
||||
|
||||
v.SetMapIndex(keyelem, valelem)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Decode a group.
|
||||
func (o *Buffer) dec_struct_group(p *Properties, base structPointer) error {
|
||||
bas := structPointer_GetStructPointer(base, p.field)
|
||||
if structPointer_IsNil(bas) {
|
||||
// allocate new nested message
|
||||
bas = toStructPointer(reflect.New(p.stype))
|
||||
structPointer_SetStructPointer(base, p.field, bas)
|
||||
}
|
||||
return o.unmarshalType(p.stype, p.sprop, true, bas)
|
||||
}
|
||||
|
||||
// Decode an embedded message.
|
||||
func (o *Buffer) dec_struct_message(p *Properties, base structPointer) (err error) {
|
||||
raw, e := o.DecodeRawBytes(false)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
|
||||
bas := structPointer_GetStructPointer(base, p.field)
|
||||
if structPointer_IsNil(bas) {
|
||||
// allocate new nested message
|
||||
bas = toStructPointer(reflect.New(p.stype))
|
||||
structPointer_SetStructPointer(base, p.field, bas)
|
||||
}
|
||||
|
||||
// If the object can unmarshal itself, let it.
|
||||
if p.isUnmarshaler {
|
||||
iv := structPointer_Interface(bas, p.stype)
|
||||
return iv.(Unmarshaler).Unmarshal(raw)
|
||||
}
|
||||
|
||||
obuf := o.buf
|
||||
oi := o.index
|
||||
o.buf = raw
|
||||
o.index = 0
|
||||
|
||||
err = o.unmarshalType(p.stype, p.sprop, false, bas)
|
||||
o.buf = obuf
|
||||
o.index = oi
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// Decode a slice of embedded messages.
|
||||
func (o *Buffer) dec_slice_struct_message(p *Properties, base structPointer) error {
|
||||
return o.dec_slice_struct(p, false, base)
|
||||
}
|
||||
|
||||
// Decode a slice of embedded groups.
|
||||
func (o *Buffer) dec_slice_struct_group(p *Properties, base structPointer) error {
|
||||
return o.dec_slice_struct(p, true, base)
|
||||
}
|
||||
|
||||
// Decode a slice of structs ([]*struct).
|
||||
func (o *Buffer) dec_slice_struct(p *Properties, is_group bool, base structPointer) error {
|
||||
v := reflect.New(p.stype)
|
||||
bas := toStructPointer(v)
|
||||
structPointer_StructPointerSlice(base, p.field).Append(bas)
|
||||
|
||||
if is_group {
|
||||
err := o.unmarshalType(p.stype, p.sprop, is_group, bas)
|
||||
return err
|
||||
}
|
||||
|
||||
raw, err := o.DecodeRawBytes(false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// If the object can unmarshal itself, let it.
|
||||
if p.isUnmarshaler {
|
||||
iv := v.Interface()
|
||||
return iv.(Unmarshaler).Unmarshal(raw)
|
||||
}
|
||||
|
||||
obuf := o.buf
|
||||
oi := o.index
|
||||
o.buf = raw
|
||||
o.index = 0
|
||||
|
||||
err = o.unmarshalType(p.stype, p.sprop, is_group, bas)
|
||||
|
||||
o.buf = obuf
|
||||
o.index = oi
|
||||
|
||||
return err
|
||||
}
|
||||
+1362
File diff suppressed because it is too large
Load Diff
+300
@@ -0,0 +1,300 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2011 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
// Protocol buffer comparison.
|
||||
|
||||
package proto
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
/*
|
||||
Equal returns true iff protocol buffers a and b are equal.
|
||||
The arguments must both be pointers to protocol buffer structs.
|
||||
|
||||
Equality is defined in this way:
|
||||
- Two messages are equal iff they are the same type,
|
||||
corresponding fields are equal, unknown field sets
|
||||
are equal, and extensions sets are equal.
|
||||
- Two set scalar fields are equal iff their values are equal.
|
||||
If the fields are of a floating-point type, remember that
|
||||
NaN != x for all x, including NaN. If the message is defined
|
||||
in a proto3 .proto file, fields are not "set"; specifically,
|
||||
zero length proto3 "bytes" fields are equal (nil == {}).
|
||||
- Two repeated fields are equal iff their lengths are the same,
|
||||
and their corresponding elements are equal. Note a "bytes" field,
|
||||
although represented by []byte, is not a repeated field and the
|
||||
rule for the scalar fields described above applies.
|
||||
- Two unset fields are equal.
|
||||
- Two unknown field sets are equal if their current
|
||||
encoded state is equal.
|
||||
- Two extension sets are equal iff they have corresponding
|
||||
elements that are pairwise equal.
|
||||
- Two map fields are equal iff their lengths are the same,
|
||||
and they contain the same set of elements. Zero-length map
|
||||
fields are equal.
|
||||
- Every other combination of things are not equal.
|
||||
|
||||
The return value is undefined if a and b are not protocol buffers.
|
||||
*/
|
||||
func Equal(a, b Message) bool {
|
||||
if a == nil || b == nil {
|
||||
return a == b
|
||||
}
|
||||
v1, v2 := reflect.ValueOf(a), reflect.ValueOf(b)
|
||||
if v1.Type() != v2.Type() {
|
||||
return false
|
||||
}
|
||||
if v1.Kind() == reflect.Ptr {
|
||||
if v1.IsNil() {
|
||||
return v2.IsNil()
|
||||
}
|
||||
if v2.IsNil() {
|
||||
return false
|
||||
}
|
||||
v1, v2 = v1.Elem(), v2.Elem()
|
||||
}
|
||||
if v1.Kind() != reflect.Struct {
|
||||
return false
|
||||
}
|
||||
return equalStruct(v1, v2)
|
||||
}
|
||||
|
||||
// v1 and v2 are known to have the same type.
|
||||
func equalStruct(v1, v2 reflect.Value) bool {
|
||||
sprop := GetProperties(v1.Type())
|
||||
for i := 0; i < v1.NumField(); i++ {
|
||||
f := v1.Type().Field(i)
|
||||
if strings.HasPrefix(f.Name, "XXX_") {
|
||||
continue
|
||||
}
|
||||
f1, f2 := v1.Field(i), v2.Field(i)
|
||||
if f.Type.Kind() == reflect.Ptr {
|
||||
if n1, n2 := f1.IsNil(), f2.IsNil(); n1 && n2 {
|
||||
// both unset
|
||||
continue
|
||||
} else if n1 != n2 {
|
||||
// set/unset mismatch
|
||||
return false
|
||||
}
|
||||
b1, ok := f1.Interface().(raw)
|
||||
if ok {
|
||||
b2 := f2.Interface().(raw)
|
||||
// RawMessage
|
||||
if !bytes.Equal(b1.Bytes(), b2.Bytes()) {
|
||||
return false
|
||||
}
|
||||
continue
|
||||
}
|
||||
f1, f2 = f1.Elem(), f2.Elem()
|
||||
}
|
||||
if !equalAny(f1, f2, sprop.Prop[i]) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
if em1 := v1.FieldByName("XXX_InternalExtensions"); em1.IsValid() {
|
||||
em2 := v2.FieldByName("XXX_InternalExtensions")
|
||||
if !equalExtensions(v1.Type(), em1.Interface().(XXX_InternalExtensions), em2.Interface().(XXX_InternalExtensions)) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
if em1 := v1.FieldByName("XXX_extensions"); em1.IsValid() {
|
||||
em2 := v2.FieldByName("XXX_extensions")
|
||||
if !equalExtMap(v1.Type(), em1.Interface().(map[int32]Extension), em2.Interface().(map[int32]Extension)) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
uf := v1.FieldByName("XXX_unrecognized")
|
||||
if !uf.IsValid() {
|
||||
return true
|
||||
}
|
||||
|
||||
u1 := uf.Bytes()
|
||||
u2 := v2.FieldByName("XXX_unrecognized").Bytes()
|
||||
if !bytes.Equal(u1, u2) {
|
||||
return false
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
// v1 and v2 are known to have the same type.
|
||||
// prop may be nil.
|
||||
func equalAny(v1, v2 reflect.Value, prop *Properties) bool {
|
||||
if v1.Type() == protoMessageType {
|
||||
m1, _ := v1.Interface().(Message)
|
||||
m2, _ := v2.Interface().(Message)
|
||||
return Equal(m1, m2)
|
||||
}
|
||||
switch v1.Kind() {
|
||||
case reflect.Bool:
|
||||
return v1.Bool() == v2.Bool()
|
||||
case reflect.Float32, reflect.Float64:
|
||||
return v1.Float() == v2.Float()
|
||||
case reflect.Int32, reflect.Int64:
|
||||
return v1.Int() == v2.Int()
|
||||
case reflect.Interface:
|
||||
// Probably a oneof field; compare the inner values.
|
||||
n1, n2 := v1.IsNil(), v2.IsNil()
|
||||
if n1 || n2 {
|
||||
return n1 == n2
|
||||
}
|
||||
e1, e2 := v1.Elem(), v2.Elem()
|
||||
if e1.Type() != e2.Type() {
|
||||
return false
|
||||
}
|
||||
return equalAny(e1, e2, nil)
|
||||
case reflect.Map:
|
||||
if v1.Len() != v2.Len() {
|
||||
return false
|
||||
}
|
||||
for _, key := range v1.MapKeys() {
|
||||
val2 := v2.MapIndex(key)
|
||||
if !val2.IsValid() {
|
||||
// This key was not found in the second map.
|
||||
return false
|
||||
}
|
||||
if !equalAny(v1.MapIndex(key), val2, nil) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
case reflect.Ptr:
|
||||
// Maps may have nil values in them, so check for nil.
|
||||
if v1.IsNil() && v2.IsNil() {
|
||||
return true
|
||||
}
|
||||
if v1.IsNil() != v2.IsNil() {
|
||||
return false
|
||||
}
|
||||
return equalAny(v1.Elem(), v2.Elem(), prop)
|
||||
case reflect.Slice:
|
||||
if v1.Type().Elem().Kind() == reflect.Uint8 {
|
||||
// short circuit: []byte
|
||||
|
||||
// Edge case: if this is in a proto3 message, a zero length
|
||||
// bytes field is considered the zero value.
|
||||
if prop != nil && prop.proto3 && v1.Len() == 0 && v2.Len() == 0 {
|
||||
return true
|
||||
}
|
||||
if v1.IsNil() != v2.IsNil() {
|
||||
return false
|
||||
}
|
||||
return bytes.Equal(v1.Interface().([]byte), v2.Interface().([]byte))
|
||||
}
|
||||
|
||||
if v1.Len() != v2.Len() {
|
||||
return false
|
||||
}
|
||||
for i := 0; i < v1.Len(); i++ {
|
||||
if !equalAny(v1.Index(i), v2.Index(i), prop) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
case reflect.String:
|
||||
return v1.Interface().(string) == v2.Interface().(string)
|
||||
case reflect.Struct:
|
||||
return equalStruct(v1, v2)
|
||||
case reflect.Uint32, reflect.Uint64:
|
||||
return v1.Uint() == v2.Uint()
|
||||
}
|
||||
|
||||
// unknown type, so not a protocol buffer
|
||||
log.Printf("proto: don't know how to compare %v", v1)
|
||||
return false
|
||||
}
|
||||
|
||||
// base is the struct type that the extensions are based on.
|
||||
// x1 and x2 are InternalExtensions.
|
||||
func equalExtensions(base reflect.Type, x1, x2 XXX_InternalExtensions) bool {
|
||||
em1, _ := x1.extensionsRead()
|
||||
em2, _ := x2.extensionsRead()
|
||||
return equalExtMap(base, em1, em2)
|
||||
}
|
||||
|
||||
func equalExtMap(base reflect.Type, em1, em2 map[int32]Extension) bool {
|
||||
if len(em1) != len(em2) {
|
||||
return false
|
||||
}
|
||||
|
||||
for extNum, e1 := range em1 {
|
||||
e2, ok := em2[extNum]
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
|
||||
m1, m2 := e1.value, e2.value
|
||||
|
||||
if m1 != nil && m2 != nil {
|
||||
// Both are unencoded.
|
||||
if !equalAny(reflect.ValueOf(m1), reflect.ValueOf(m2), nil) {
|
||||
return false
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// At least one is encoded. To do a semantically correct comparison
|
||||
// we need to unmarshal them first.
|
||||
var desc *ExtensionDesc
|
||||
if m := extensionMaps[base]; m != nil {
|
||||
desc = m[extNum]
|
||||
}
|
||||
if desc == nil {
|
||||
log.Printf("proto: don't know how to compare extension %d of %v", extNum, base)
|
||||
continue
|
||||
}
|
||||
var err error
|
||||
if m1 == nil {
|
||||
m1, err = decodeExtension(e1.enc, desc)
|
||||
}
|
||||
if m2 == nil && err == nil {
|
||||
m2, err = decodeExtension(e2.enc, desc)
|
||||
}
|
||||
if err != nil {
|
||||
// The encoded form is invalid.
|
||||
log.Printf("proto: badly encoded extension %d of %v: %v", extNum, base, err)
|
||||
return false
|
||||
}
|
||||
if !equalAny(reflect.ValueOf(m1), reflect.ValueOf(m2), nil) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
+587
@@ -0,0 +1,587 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
package proto
|
||||
|
||||
/*
|
||||
* Types and routines for supporting protocol buffer extensions.
|
||||
*/
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// ErrMissingExtension is the error returned by GetExtension if the named extension is not in the message.
|
||||
var ErrMissingExtension = errors.New("proto: missing extension")
|
||||
|
||||
// ExtensionRange represents a range of message extensions for a protocol buffer.
|
||||
// Used in code generated by the protocol compiler.
|
||||
type ExtensionRange struct {
|
||||
Start, End int32 // both inclusive
|
||||
}
|
||||
|
||||
// extendableProto is an interface implemented by any protocol buffer generated by the current
|
||||
// proto compiler that may be extended.
|
||||
type extendableProto interface {
|
||||
Message
|
||||
ExtensionRangeArray() []ExtensionRange
|
||||
extensionsWrite() map[int32]Extension
|
||||
extensionsRead() (map[int32]Extension, sync.Locker)
|
||||
}
|
||||
|
||||
// extendableProtoV1 is an interface implemented by a protocol buffer generated by the previous
|
||||
// version of the proto compiler that may be extended.
|
||||
type extendableProtoV1 interface {
|
||||
Message
|
||||
ExtensionRangeArray() []ExtensionRange
|
||||
ExtensionMap() map[int32]Extension
|
||||
}
|
||||
|
||||
// extensionAdapter is a wrapper around extendableProtoV1 that implements extendableProto.
|
||||
type extensionAdapter struct {
|
||||
extendableProtoV1
|
||||
}
|
||||
|
||||
func (e extensionAdapter) extensionsWrite() map[int32]Extension {
|
||||
return e.ExtensionMap()
|
||||
}
|
||||
|
||||
func (e extensionAdapter) extensionsRead() (map[int32]Extension, sync.Locker) {
|
||||
return e.ExtensionMap(), notLocker{}
|
||||
}
|
||||
|
||||
// notLocker is a sync.Locker whose Lock and Unlock methods are nops.
|
||||
type notLocker struct{}
|
||||
|
||||
func (n notLocker) Lock() {}
|
||||
func (n notLocker) Unlock() {}
|
||||
|
||||
// extendable returns the extendableProto interface for the given generated proto message.
|
||||
// If the proto message has the old extension format, it returns a wrapper that implements
|
||||
// the extendableProto interface.
|
||||
func extendable(p interface{}) (extendableProto, bool) {
|
||||
if ep, ok := p.(extendableProto); ok {
|
||||
return ep, ok
|
||||
}
|
||||
if ep, ok := p.(extendableProtoV1); ok {
|
||||
return extensionAdapter{ep}, ok
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// XXX_InternalExtensions is an internal representation of proto extensions.
|
||||
//
|
||||
// Each generated message struct type embeds an anonymous XXX_InternalExtensions field,
|
||||
// thus gaining the unexported 'extensions' method, which can be called only from the proto package.
|
||||
//
|
||||
// The methods of XXX_InternalExtensions are not concurrency safe in general,
|
||||
// but calls to logically read-only methods such as has and get may be executed concurrently.
|
||||
type XXX_InternalExtensions struct {
|
||||
// The struct must be indirect so that if a user inadvertently copies a
|
||||
// generated message and its embedded XXX_InternalExtensions, they
|
||||
// avoid the mayhem of a copied mutex.
|
||||
//
|
||||
// The mutex serializes all logically read-only operations to p.extensionMap.
|
||||
// It is up to the client to ensure that write operations to p.extensionMap are
|
||||
// mutually exclusive with other accesses.
|
||||
p *struct {
|
||||
mu sync.Mutex
|
||||
extensionMap map[int32]Extension
|
||||
}
|
||||
}
|
||||
|
||||
// extensionsWrite returns the extension map, creating it on first use.
|
||||
func (e *XXX_InternalExtensions) extensionsWrite() map[int32]Extension {
|
||||
if e.p == nil {
|
||||
e.p = new(struct {
|
||||
mu sync.Mutex
|
||||
extensionMap map[int32]Extension
|
||||
})
|
||||
e.p.extensionMap = make(map[int32]Extension)
|
||||
}
|
||||
return e.p.extensionMap
|
||||
}
|
||||
|
||||
// extensionsRead returns the extensions map for read-only use. It may be nil.
|
||||
// The caller must hold the returned mutex's lock when accessing Elements within the map.
|
||||
func (e *XXX_InternalExtensions) extensionsRead() (map[int32]Extension, sync.Locker) {
|
||||
if e.p == nil {
|
||||
return nil, nil
|
||||
}
|
||||
return e.p.extensionMap, &e.p.mu
|
||||
}
|
||||
|
||||
var extendableProtoType = reflect.TypeOf((*extendableProto)(nil)).Elem()
|
||||
var extendableProtoV1Type = reflect.TypeOf((*extendableProtoV1)(nil)).Elem()
|
||||
|
||||
// ExtensionDesc represents an extension specification.
|
||||
// Used in generated code from the protocol compiler.
|
||||
type ExtensionDesc struct {
|
||||
ExtendedType Message // nil pointer to the type that is being extended
|
||||
ExtensionType interface{} // nil pointer to the extension type
|
||||
Field int32 // field number
|
||||
Name string // fully-qualified name of extension, for text formatting
|
||||
Tag string // protobuf tag style
|
||||
Filename string // name of the file in which the extension is defined
|
||||
}
|
||||
|
||||
func (ed *ExtensionDesc) repeated() bool {
|
||||
t := reflect.TypeOf(ed.ExtensionType)
|
||||
return t.Kind() == reflect.Slice && t.Elem().Kind() != reflect.Uint8
|
||||
}
|
||||
|
||||
// Extension represents an extension in a message.
|
||||
type Extension struct {
|
||||
// When an extension is stored in a message using SetExtension
|
||||
// only desc and value are set. When the message is marshaled
|
||||
// enc will be set to the encoded form of the message.
|
||||
//
|
||||
// When a message is unmarshaled and contains extensions, each
|
||||
// extension will have only enc set. When such an extension is
|
||||
// accessed using GetExtension (or GetExtensions) desc and value
|
||||
// will be set.
|
||||
desc *ExtensionDesc
|
||||
value interface{}
|
||||
enc []byte
|
||||
}
|
||||
|
||||
// SetRawExtension is for testing only.
|
||||
func SetRawExtension(base Message, id int32, b []byte) {
|
||||
epb, ok := extendable(base)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
extmap := epb.extensionsWrite()
|
||||
extmap[id] = Extension{enc: b}
|
||||
}
|
||||
|
||||
// isExtensionField returns true iff the given field number is in an extension range.
|
||||
func isExtensionField(pb extendableProto, field int32) bool {
|
||||
for _, er := range pb.ExtensionRangeArray() {
|
||||
if er.Start <= field && field <= er.End {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// checkExtensionTypes checks that the given extension is valid for pb.
|
||||
func checkExtensionTypes(pb extendableProto, extension *ExtensionDesc) error {
|
||||
var pbi interface{} = pb
|
||||
// Check the extended type.
|
||||
if ea, ok := pbi.(extensionAdapter); ok {
|
||||
pbi = ea.extendableProtoV1
|
||||
}
|
||||
if a, b := reflect.TypeOf(pbi), reflect.TypeOf(extension.ExtendedType); a != b {
|
||||
return errors.New("proto: bad extended type; " + b.String() + " does not extend " + a.String())
|
||||
}
|
||||
// Check the range.
|
||||
if !isExtensionField(pb, extension.Field) {
|
||||
return errors.New("proto: bad extension number; not in declared ranges")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// extPropKey is sufficient to uniquely identify an extension.
|
||||
type extPropKey struct {
|
||||
base reflect.Type
|
||||
field int32
|
||||
}
|
||||
|
||||
var extProp = struct {
|
||||
sync.RWMutex
|
||||
m map[extPropKey]*Properties
|
||||
}{
|
||||
m: make(map[extPropKey]*Properties),
|
||||
}
|
||||
|
||||
func extensionProperties(ed *ExtensionDesc) *Properties {
|
||||
key := extPropKey{base: reflect.TypeOf(ed.ExtendedType), field: ed.Field}
|
||||
|
||||
extProp.RLock()
|
||||
if prop, ok := extProp.m[key]; ok {
|
||||
extProp.RUnlock()
|
||||
return prop
|
||||
}
|
||||
extProp.RUnlock()
|
||||
|
||||
extProp.Lock()
|
||||
defer extProp.Unlock()
|
||||
// Check again.
|
||||
if prop, ok := extProp.m[key]; ok {
|
||||
return prop
|
||||
}
|
||||
|
||||
prop := new(Properties)
|
||||
prop.Init(reflect.TypeOf(ed.ExtensionType), "unknown_name", ed.Tag, nil)
|
||||
extProp.m[key] = prop
|
||||
return prop
|
||||
}
|
||||
|
||||
// encode encodes any unmarshaled (unencoded) extensions in e.
|
||||
func encodeExtensions(e *XXX_InternalExtensions) error {
|
||||
m, mu := e.extensionsRead()
|
||||
if m == nil {
|
||||
return nil // fast path
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return encodeExtensionsMap(m)
|
||||
}
|
||||
|
||||
// encode encodes any unmarshaled (unencoded) extensions in e.
|
||||
func encodeExtensionsMap(m map[int32]Extension) error {
|
||||
for k, e := range m {
|
||||
if e.value == nil || e.desc == nil {
|
||||
// Extension is only in its encoded form.
|
||||
continue
|
||||
}
|
||||
|
||||
// We don't skip extensions that have an encoded form set,
|
||||
// because the extension value may have been mutated after
|
||||
// the last time this function was called.
|
||||
|
||||
et := reflect.TypeOf(e.desc.ExtensionType)
|
||||
props := extensionProperties(e.desc)
|
||||
|
||||
p := NewBuffer(nil)
|
||||
// If e.value has type T, the encoder expects a *struct{ X T }.
|
||||
// Pass a *T with a zero field and hope it all works out.
|
||||
x := reflect.New(et)
|
||||
x.Elem().Set(reflect.ValueOf(e.value))
|
||||
if err := props.enc(p, props, toStructPointer(x)); err != nil {
|
||||
return err
|
||||
}
|
||||
e.enc = p.buf
|
||||
m[k] = e
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func extensionsSize(e *XXX_InternalExtensions) (n int) {
|
||||
m, mu := e.extensionsRead()
|
||||
if m == nil {
|
||||
return 0
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return extensionsMapSize(m)
|
||||
}
|
||||
|
||||
func extensionsMapSize(m map[int32]Extension) (n int) {
|
||||
for _, e := range m {
|
||||
if e.value == nil || e.desc == nil {
|
||||
// Extension is only in its encoded form.
|
||||
n += len(e.enc)
|
||||
continue
|
||||
}
|
||||
|
||||
// We don't skip extensions that have an encoded form set,
|
||||
// because the extension value may have been mutated after
|
||||
// the last time this function was called.
|
||||
|
||||
et := reflect.TypeOf(e.desc.ExtensionType)
|
||||
props := extensionProperties(e.desc)
|
||||
|
||||
// If e.value has type T, the encoder expects a *struct{ X T }.
|
||||
// Pass a *T with a zero field and hope it all works out.
|
||||
x := reflect.New(et)
|
||||
x.Elem().Set(reflect.ValueOf(e.value))
|
||||
n += props.size(props, toStructPointer(x))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// HasExtension returns whether the given extension is present in pb.
|
||||
func HasExtension(pb Message, extension *ExtensionDesc) bool {
|
||||
// TODO: Check types, field numbers, etc.?
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
extmap, mu := epb.extensionsRead()
|
||||
if extmap == nil {
|
||||
return false
|
||||
}
|
||||
mu.Lock()
|
||||
_, ok = extmap[extension.Field]
|
||||
mu.Unlock()
|
||||
return ok
|
||||
}
|
||||
|
||||
// ClearExtension removes the given extension from pb.
|
||||
func ClearExtension(pb Message, extension *ExtensionDesc) {
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// TODO: Check types, field numbers, etc.?
|
||||
extmap := epb.extensionsWrite()
|
||||
delete(extmap, extension.Field)
|
||||
}
|
||||
|
||||
// GetExtension parses and returns the given extension of pb.
|
||||
// If the extension is not present and has no default value it returns ErrMissingExtension.
|
||||
func GetExtension(pb Message, extension *ExtensionDesc) (interface{}, error) {
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return nil, errors.New("proto: not an extendable proto")
|
||||
}
|
||||
|
||||
if err := checkExtensionTypes(epb, extension); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
emap, mu := epb.extensionsRead()
|
||||
if emap == nil {
|
||||
return defaultExtensionValue(extension)
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
e, ok := emap[extension.Field]
|
||||
if !ok {
|
||||
// defaultExtensionValue returns the default value or
|
||||
// ErrMissingExtension if there is no default.
|
||||
return defaultExtensionValue(extension)
|
||||
}
|
||||
|
||||
if e.value != nil {
|
||||
// Already decoded. Check the descriptor, though.
|
||||
if e.desc != extension {
|
||||
// This shouldn't happen. If it does, it means that
|
||||
// GetExtension was called twice with two different
|
||||
// descriptors with the same field number.
|
||||
return nil, errors.New("proto: descriptor conflict")
|
||||
}
|
||||
return e.value, nil
|
||||
}
|
||||
|
||||
v, err := decodeExtension(e.enc, extension)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Remember the decoded version and drop the encoded version.
|
||||
// That way it is safe to mutate what we return.
|
||||
e.value = v
|
||||
e.desc = extension
|
||||
e.enc = nil
|
||||
emap[extension.Field] = e
|
||||
return e.value, nil
|
||||
}
|
||||
|
||||
// defaultExtensionValue returns the default value for extension.
|
||||
// If no default for an extension is defined ErrMissingExtension is returned.
|
||||
func defaultExtensionValue(extension *ExtensionDesc) (interface{}, error) {
|
||||
t := reflect.TypeOf(extension.ExtensionType)
|
||||
props := extensionProperties(extension)
|
||||
|
||||
sf, _, err := fieldDefault(t, props)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if sf == nil || sf.value == nil {
|
||||
// There is no default value.
|
||||
return nil, ErrMissingExtension
|
||||
}
|
||||
|
||||
if t.Kind() != reflect.Ptr {
|
||||
// We do not need to return a Ptr, we can directly return sf.value.
|
||||
return sf.value, nil
|
||||
}
|
||||
|
||||
// We need to return an interface{} that is a pointer to sf.value.
|
||||
value := reflect.New(t).Elem()
|
||||
value.Set(reflect.New(value.Type().Elem()))
|
||||
if sf.kind == reflect.Int32 {
|
||||
// We may have an int32 or an enum, but the underlying data is int32.
|
||||
// Since we can't set an int32 into a non int32 reflect.value directly
|
||||
// set it as a int32.
|
||||
value.Elem().SetInt(int64(sf.value.(int32)))
|
||||
} else {
|
||||
value.Elem().Set(reflect.ValueOf(sf.value))
|
||||
}
|
||||
return value.Interface(), nil
|
||||
}
|
||||
|
||||
// decodeExtension decodes an extension encoded in b.
|
||||
func decodeExtension(b []byte, extension *ExtensionDesc) (interface{}, error) {
|
||||
o := NewBuffer(b)
|
||||
|
||||
t := reflect.TypeOf(extension.ExtensionType)
|
||||
|
||||
props := extensionProperties(extension)
|
||||
|
||||
// t is a pointer to a struct, pointer to basic type or a slice.
|
||||
// Allocate a "field" to store the pointer/slice itself; the
|
||||
// pointer/slice will be stored here. We pass
|
||||
// the address of this field to props.dec.
|
||||
// This passes a zero field and a *t and lets props.dec
|
||||
// interpret it as a *struct{ x t }.
|
||||
value := reflect.New(t).Elem()
|
||||
|
||||
for {
|
||||
// Discard wire type and field number varint. It isn't needed.
|
||||
if _, err := o.DecodeVarint(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := props.dec(o, props, toStructPointer(value.Addr())); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if o.index >= len(o.buf) {
|
||||
break
|
||||
}
|
||||
}
|
||||
return value.Interface(), nil
|
||||
}
|
||||
|
||||
// GetExtensions returns a slice of the extensions present in pb that are also listed in es.
|
||||
// The returned slice has the same length as es; missing extensions will appear as nil elements.
|
||||
func GetExtensions(pb Message, es []*ExtensionDesc) (extensions []interface{}, err error) {
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return nil, errors.New("proto: not an extendable proto")
|
||||
}
|
||||
extensions = make([]interface{}, len(es))
|
||||
for i, e := range es {
|
||||
extensions[i], err = GetExtension(epb, e)
|
||||
if err == ErrMissingExtension {
|
||||
err = nil
|
||||
}
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// ExtensionDescs returns a new slice containing pb's extension descriptors, in undefined order.
|
||||
// For non-registered extensions, ExtensionDescs returns an incomplete descriptor containing
|
||||
// just the Field field, which defines the extension's field number.
|
||||
func ExtensionDescs(pb Message) ([]*ExtensionDesc, error) {
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("proto: %T is not an extendable proto.Message", pb)
|
||||
}
|
||||
registeredExtensions := RegisteredExtensions(pb)
|
||||
|
||||
emap, mu := epb.extensionsRead()
|
||||
if emap == nil {
|
||||
return nil, nil
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
extensions := make([]*ExtensionDesc, 0, len(emap))
|
||||
for extid, e := range emap {
|
||||
desc := e.desc
|
||||
if desc == nil {
|
||||
desc = registeredExtensions[extid]
|
||||
if desc == nil {
|
||||
desc = &ExtensionDesc{Field: extid}
|
||||
}
|
||||
}
|
||||
|
||||
extensions = append(extensions, desc)
|
||||
}
|
||||
return extensions, nil
|
||||
}
|
||||
|
||||
// SetExtension sets the specified extension of pb to the specified value.
|
||||
func SetExtension(pb Message, extension *ExtensionDesc, value interface{}) error {
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return errors.New("proto: not an extendable proto")
|
||||
}
|
||||
if err := checkExtensionTypes(epb, extension); err != nil {
|
||||
return err
|
||||
}
|
||||
typ := reflect.TypeOf(extension.ExtensionType)
|
||||
if typ != reflect.TypeOf(value) {
|
||||
return errors.New("proto: bad extension value type")
|
||||
}
|
||||
// nil extension values need to be caught early, because the
|
||||
// encoder can't distinguish an ErrNil due to a nil extension
|
||||
// from an ErrNil due to a missing field. Extensions are
|
||||
// always optional, so the encoder would just swallow the error
|
||||
// and drop all the extensions from the encoded message.
|
||||
if reflect.ValueOf(value).IsNil() {
|
||||
return fmt.Errorf("proto: SetExtension called with nil value of type %T", value)
|
||||
}
|
||||
|
||||
extmap := epb.extensionsWrite()
|
||||
extmap[extension.Field] = Extension{desc: extension, value: value}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ClearAllExtensions clears all extensions from pb.
|
||||
func ClearAllExtensions(pb Message) {
|
||||
epb, ok := extendable(pb)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
m := epb.extensionsWrite()
|
||||
for k := range m {
|
||||
delete(m, k)
|
||||
}
|
||||
}
|
||||
|
||||
// A global registry of extensions.
|
||||
// The generated code will register the generated descriptors by calling RegisterExtension.
|
||||
|
||||
var extensionMaps = make(map[reflect.Type]map[int32]*ExtensionDesc)
|
||||
|
||||
// RegisterExtension is called from the generated code.
|
||||
func RegisterExtension(desc *ExtensionDesc) {
|
||||
st := reflect.TypeOf(desc.ExtendedType).Elem()
|
||||
m := extensionMaps[st]
|
||||
if m == nil {
|
||||
m = make(map[int32]*ExtensionDesc)
|
||||
extensionMaps[st] = m
|
||||
}
|
||||
if _, ok := m[desc.Field]; ok {
|
||||
panic("proto: duplicate extension registered: " + st.String() + " " + strconv.Itoa(int(desc.Field)))
|
||||
}
|
||||
m[desc.Field] = desc
|
||||
}
|
||||
|
||||
// RegisteredExtensions returns a map of the registered extensions of a
|
||||
// protocol buffer struct, indexed by the extension number.
|
||||
// The argument pb should be a nil pointer to the struct type.
|
||||
func RegisteredExtensions(pb Message) map[int32]*ExtensionDesc {
|
||||
return extensionMaps[reflect.TypeOf(pb).Elem()]
|
||||
}
|
||||
+897
@@ -0,0 +1,897 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
/*
|
||||
Package proto converts data structures to and from the wire format of
|
||||
protocol buffers. It works in concert with the Go source code generated
|
||||
for .proto files by the protocol compiler.
|
||||
|
||||
A summary of the properties of the protocol buffer interface
|
||||
for a protocol buffer variable v:
|
||||
|
||||
- Names are turned from camel_case to CamelCase for export.
|
||||
- There are no methods on v to set fields; just treat
|
||||
them as structure fields.
|
||||
- There are getters that return a field's value if set,
|
||||
and return the field's default value if unset.
|
||||
The getters work even if the receiver is a nil message.
|
||||
- The zero value for a struct is its correct initialization state.
|
||||
All desired fields must be set before marshaling.
|
||||
- A Reset() method will restore a protobuf struct to its zero state.
|
||||
- Non-repeated fields are pointers to the values; nil means unset.
|
||||
That is, optional or required field int32 f becomes F *int32.
|
||||
- Repeated fields are slices.
|
||||
- Helper functions are available to aid the setting of fields.
|
||||
msg.Foo = proto.String("hello") // set field
|
||||
- Constants are defined to hold the default values of all fields that
|
||||
have them. They have the form Default_StructName_FieldName.
|
||||
Because the getter methods handle defaulted values,
|
||||
direct use of these constants should be rare.
|
||||
- Enums are given type names and maps from names to values.
|
||||
Enum values are prefixed by the enclosing message's name, or by the
|
||||
enum's type name if it is a top-level enum. Enum types have a String
|
||||
method, and a Enum method to assist in message construction.
|
||||
- Nested messages, groups and enums have type names prefixed with the name of
|
||||
the surrounding message type.
|
||||
- Extensions are given descriptor names that start with E_,
|
||||
followed by an underscore-delimited list of the nested messages
|
||||
that contain it (if any) followed by the CamelCased name of the
|
||||
extension field itself. HasExtension, ClearExtension, GetExtension
|
||||
and SetExtension are functions for manipulating extensions.
|
||||
- Oneof field sets are given a single field in their message,
|
||||
with distinguished wrapper types for each possible field value.
|
||||
- Marshal and Unmarshal are functions to encode and decode the wire format.
|
||||
|
||||
When the .proto file specifies `syntax="proto3"`, there are some differences:
|
||||
|
||||
- Non-repeated fields of non-message type are values instead of pointers.
|
||||
- Enum types do not get an Enum method.
|
||||
|
||||
The simplest way to describe this is to see an example.
|
||||
Given file test.proto, containing
|
||||
|
||||
package example;
|
||||
|
||||
enum FOO { X = 17; }
|
||||
|
||||
message Test {
|
||||
required string label = 1;
|
||||
optional int32 type = 2 [default=77];
|
||||
repeated int64 reps = 3;
|
||||
optional group OptionalGroup = 4 {
|
||||
required string RequiredField = 5;
|
||||
}
|
||||
oneof union {
|
||||
int32 number = 6;
|
||||
string name = 7;
|
||||
}
|
||||
}
|
||||
|
||||
The resulting file, test.pb.go, is:
|
||||
|
||||
package example
|
||||
|
||||
import proto "github.com/golang/protobuf/proto"
|
||||
import math "math"
|
||||
|
||||
type FOO int32
|
||||
const (
|
||||
FOO_X FOO = 17
|
||||
)
|
||||
var FOO_name = map[int32]string{
|
||||
17: "X",
|
||||
}
|
||||
var FOO_value = map[string]int32{
|
||||
"X": 17,
|
||||
}
|
||||
|
||||
func (x FOO) Enum() *FOO {
|
||||
p := new(FOO)
|
||||
*p = x
|
||||
return p
|
||||
}
|
||||
func (x FOO) String() string {
|
||||
return proto.EnumName(FOO_name, int32(x))
|
||||
}
|
||||
func (x *FOO) UnmarshalJSON(data []byte) error {
|
||||
value, err := proto.UnmarshalJSONEnum(FOO_value, data)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*x = FOO(value)
|
||||
return nil
|
||||
}
|
||||
|
||||
type Test struct {
|
||||
Label *string `protobuf:"bytes,1,req,name=label" json:"label,omitempty"`
|
||||
Type *int32 `protobuf:"varint,2,opt,name=type,def=77" json:"type,omitempty"`
|
||||
Reps []int64 `protobuf:"varint,3,rep,name=reps" json:"reps,omitempty"`
|
||||
Optionalgroup *Test_OptionalGroup `protobuf:"group,4,opt,name=OptionalGroup" json:"optionalgroup,omitempty"`
|
||||
// Types that are valid to be assigned to Union:
|
||||
// *Test_Number
|
||||
// *Test_Name
|
||||
Union isTest_Union `protobuf_oneof:"union"`
|
||||
XXX_unrecognized []byte `json:"-"`
|
||||
}
|
||||
func (m *Test) Reset() { *m = Test{} }
|
||||
func (m *Test) String() string { return proto.CompactTextString(m) }
|
||||
func (*Test) ProtoMessage() {}
|
||||
|
||||
type isTest_Union interface {
|
||||
isTest_Union()
|
||||
}
|
||||
|
||||
type Test_Number struct {
|
||||
Number int32 `protobuf:"varint,6,opt,name=number"`
|
||||
}
|
||||
type Test_Name struct {
|
||||
Name string `protobuf:"bytes,7,opt,name=name"`
|
||||
}
|
||||
|
||||
func (*Test_Number) isTest_Union() {}
|
||||
func (*Test_Name) isTest_Union() {}
|
||||
|
||||
func (m *Test) GetUnion() isTest_Union {
|
||||
if m != nil {
|
||||
return m.Union
|
||||
}
|
||||
return nil
|
||||
}
|
||||
const Default_Test_Type int32 = 77
|
||||
|
||||
func (m *Test) GetLabel() string {
|
||||
if m != nil && m.Label != nil {
|
||||
return *m.Label
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (m *Test) GetType() int32 {
|
||||
if m != nil && m.Type != nil {
|
||||
return *m.Type
|
||||
}
|
||||
return Default_Test_Type
|
||||
}
|
||||
|
||||
func (m *Test) GetOptionalgroup() *Test_OptionalGroup {
|
||||
if m != nil {
|
||||
return m.Optionalgroup
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type Test_OptionalGroup struct {
|
||||
RequiredField *string `protobuf:"bytes,5,req" json:"RequiredField,omitempty"`
|
||||
}
|
||||
func (m *Test_OptionalGroup) Reset() { *m = Test_OptionalGroup{} }
|
||||
func (m *Test_OptionalGroup) String() string { return proto.CompactTextString(m) }
|
||||
|
||||
func (m *Test_OptionalGroup) GetRequiredField() string {
|
||||
if m != nil && m.RequiredField != nil {
|
||||
return *m.RequiredField
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (m *Test) GetNumber() int32 {
|
||||
if x, ok := m.GetUnion().(*Test_Number); ok {
|
||||
return x.Number
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func (m *Test) GetName() string {
|
||||
if x, ok := m.GetUnion().(*Test_Name); ok {
|
||||
return x.Name
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func init() {
|
||||
proto.RegisterEnum("example.FOO", FOO_name, FOO_value)
|
||||
}
|
||||
|
||||
To create and play with a Test object:
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
|
||||
"github.com/golang/protobuf/proto"
|
||||
pb "./example.pb"
|
||||
)
|
||||
|
||||
func main() {
|
||||
test := &pb.Test{
|
||||
Label: proto.String("hello"),
|
||||
Type: proto.Int32(17),
|
||||
Reps: []int64{1, 2, 3},
|
||||
Optionalgroup: &pb.Test_OptionalGroup{
|
||||
RequiredField: proto.String("good bye"),
|
||||
},
|
||||
Union: &pb.Test_Name{"fred"},
|
||||
}
|
||||
data, err := proto.Marshal(test)
|
||||
if err != nil {
|
||||
log.Fatal("marshaling error: ", err)
|
||||
}
|
||||
newTest := &pb.Test{}
|
||||
err = proto.Unmarshal(data, newTest)
|
||||
if err != nil {
|
||||
log.Fatal("unmarshaling error: ", err)
|
||||
}
|
||||
// Now test and newTest contain the same data.
|
||||
if test.GetLabel() != newTest.GetLabel() {
|
||||
log.Fatalf("data mismatch %q != %q", test.GetLabel(), newTest.GetLabel())
|
||||
}
|
||||
// Use a type switch to determine which oneof was set.
|
||||
switch u := test.Union.(type) {
|
||||
case *pb.Test_Number: // u.Number contains the number.
|
||||
case *pb.Test_Name: // u.Name contains the string.
|
||||
}
|
||||
// etc.
|
||||
}
|
||||
*/
|
||||
package proto
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strconv"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Message is implemented by generated protocol buffer messages.
|
||||
type Message interface {
|
||||
Reset()
|
||||
String() string
|
||||
ProtoMessage()
|
||||
}
|
||||
|
||||
// Stats records allocation details about the protocol buffer encoders
|
||||
// and decoders. Useful for tuning the library itself.
|
||||
type Stats struct {
|
||||
Emalloc uint64 // mallocs in encode
|
||||
Dmalloc uint64 // mallocs in decode
|
||||
Encode uint64 // number of encodes
|
||||
Decode uint64 // number of decodes
|
||||
Chit uint64 // number of cache hits
|
||||
Cmiss uint64 // number of cache misses
|
||||
Size uint64 // number of sizes
|
||||
}
|
||||
|
||||
// Set to true to enable stats collection.
|
||||
const collectStats = false
|
||||
|
||||
var stats Stats
|
||||
|
||||
// GetStats returns a copy of the global Stats structure.
|
||||
func GetStats() Stats { return stats }
|
||||
|
||||
// A Buffer is a buffer manager for marshaling and unmarshaling
|
||||
// protocol buffers. It may be reused between invocations to
|
||||
// reduce memory usage. It is not necessary to use a Buffer;
|
||||
// the global functions Marshal and Unmarshal create a
|
||||
// temporary Buffer and are fine for most applications.
|
||||
type Buffer struct {
|
||||
buf []byte // encode/decode byte stream
|
||||
index int // read point
|
||||
|
||||
// pools of basic types to amortize allocation.
|
||||
bools []bool
|
||||
uint32s []uint32
|
||||
uint64s []uint64
|
||||
|
||||
// extra pools, only used with pointer_reflect.go
|
||||
int32s []int32
|
||||
int64s []int64
|
||||
float32s []float32
|
||||
float64s []float64
|
||||
}
|
||||
|
||||
// NewBuffer allocates a new Buffer and initializes its internal data to
|
||||
// the contents of the argument slice.
|
||||
func NewBuffer(e []byte) *Buffer {
|
||||
return &Buffer{buf: e}
|
||||
}
|
||||
|
||||
// Reset resets the Buffer, ready for marshaling a new protocol buffer.
|
||||
func (p *Buffer) Reset() {
|
||||
p.buf = p.buf[0:0] // for reading/writing
|
||||
p.index = 0 // for reading
|
||||
}
|
||||
|
||||
// SetBuf replaces the internal buffer with the slice,
|
||||
// ready for unmarshaling the contents of the slice.
|
||||
func (p *Buffer) SetBuf(s []byte) {
|
||||
p.buf = s
|
||||
p.index = 0
|
||||
}
|
||||
|
||||
// Bytes returns the contents of the Buffer.
|
||||
func (p *Buffer) Bytes() []byte { return p.buf }
|
||||
|
||||
/*
|
||||
* Helper routines for simplifying the creation of optional fields of basic type.
|
||||
*/
|
||||
|
||||
// Bool is a helper routine that allocates a new bool value
|
||||
// to store v and returns a pointer to it.
|
||||
func Bool(v bool) *bool {
|
||||
return &v
|
||||
}
|
||||
|
||||
// Int32 is a helper routine that allocates a new int32 value
|
||||
// to store v and returns a pointer to it.
|
||||
func Int32(v int32) *int32 {
|
||||
return &v
|
||||
}
|
||||
|
||||
// Int is a helper routine that allocates a new int32 value
|
||||
// to store v and returns a pointer to it, but unlike Int32
|
||||
// its argument value is an int.
|
||||
func Int(v int) *int32 {
|
||||
p := new(int32)
|
||||
*p = int32(v)
|
||||
return p
|
||||
}
|
||||
|
||||
// Int64 is a helper routine that allocates a new int64 value
|
||||
// to store v and returns a pointer to it.
|
||||
func Int64(v int64) *int64 {
|
||||
return &v
|
||||
}
|
||||
|
||||
// Float32 is a helper routine that allocates a new float32 value
|
||||
// to store v and returns a pointer to it.
|
||||
func Float32(v float32) *float32 {
|
||||
return &v
|
||||
}
|
||||
|
||||
// Float64 is a helper routine that allocates a new float64 value
|
||||
// to store v and returns a pointer to it.
|
||||
func Float64(v float64) *float64 {
|
||||
return &v
|
||||
}
|
||||
|
||||
// Uint32 is a helper routine that allocates a new uint32 value
|
||||
// to store v and returns a pointer to it.
|
||||
func Uint32(v uint32) *uint32 {
|
||||
return &v
|
||||
}
|
||||
|
||||
// Uint64 is a helper routine that allocates a new uint64 value
|
||||
// to store v and returns a pointer to it.
|
||||
func Uint64(v uint64) *uint64 {
|
||||
return &v
|
||||
}
|
||||
|
||||
// String is a helper routine that allocates a new string value
|
||||
// to store v and returns a pointer to it.
|
||||
func String(v string) *string {
|
||||
return &v
|
||||
}
|
||||
|
||||
// EnumName is a helper function to simplify printing protocol buffer enums
|
||||
// by name. Given an enum map and a value, it returns a useful string.
|
||||
func EnumName(m map[int32]string, v int32) string {
|
||||
s, ok := m[v]
|
||||
if ok {
|
||||
return s
|
||||
}
|
||||
return strconv.Itoa(int(v))
|
||||
}
|
||||
|
||||
// UnmarshalJSONEnum is a helper function to simplify recovering enum int values
|
||||
// from their JSON-encoded representation. Given a map from the enum's symbolic
|
||||
// names to its int values, and a byte buffer containing the JSON-encoded
|
||||
// value, it returns an int32 that can be cast to the enum type by the caller.
|
||||
//
|
||||
// The function can deal with both JSON representations, numeric and symbolic.
|
||||
func UnmarshalJSONEnum(m map[string]int32, data []byte, enumName string) (int32, error) {
|
||||
if data[0] == '"' {
|
||||
// New style: enums are strings.
|
||||
var repr string
|
||||
if err := json.Unmarshal(data, &repr); err != nil {
|
||||
return -1, err
|
||||
}
|
||||
val, ok := m[repr]
|
||||
if !ok {
|
||||
return 0, fmt.Errorf("unrecognized enum %s value %q", enumName, repr)
|
||||
}
|
||||
return val, nil
|
||||
}
|
||||
// Old style: enums are ints.
|
||||
var val int32
|
||||
if err := json.Unmarshal(data, &val); err != nil {
|
||||
return 0, fmt.Errorf("cannot unmarshal %#q into enum %s", data, enumName)
|
||||
}
|
||||
return val, nil
|
||||
}
|
||||
|
||||
// DebugPrint dumps the encoded data in b in a debugging format with a header
|
||||
// including the string s. Used in testing but made available for general debugging.
|
||||
func (p *Buffer) DebugPrint(s string, b []byte) {
|
||||
var u uint64
|
||||
|
||||
obuf := p.buf
|
||||
index := p.index
|
||||
p.buf = b
|
||||
p.index = 0
|
||||
depth := 0
|
||||
|
||||
fmt.Printf("\n--- %s ---\n", s)
|
||||
|
||||
out:
|
||||
for {
|
||||
for i := 0; i < depth; i++ {
|
||||
fmt.Print(" ")
|
||||
}
|
||||
|
||||
index := p.index
|
||||
if index == len(p.buf) {
|
||||
break
|
||||
}
|
||||
|
||||
op, err := p.DecodeVarint()
|
||||
if err != nil {
|
||||
fmt.Printf("%3d: fetching op err %v\n", index, err)
|
||||
break out
|
||||
}
|
||||
tag := op >> 3
|
||||
wire := op & 7
|
||||
|
||||
switch wire {
|
||||
default:
|
||||
fmt.Printf("%3d: t=%3d unknown wire=%d\n",
|
||||
index, tag, wire)
|
||||
break out
|
||||
|
||||
case WireBytes:
|
||||
var r []byte
|
||||
|
||||
r, err = p.DecodeRawBytes(false)
|
||||
if err != nil {
|
||||
break out
|
||||
}
|
||||
fmt.Printf("%3d: t=%3d bytes [%d]", index, tag, len(r))
|
||||
if len(r) <= 6 {
|
||||
for i := 0; i < len(r); i++ {
|
||||
fmt.Printf(" %.2x", r[i])
|
||||
}
|
||||
} else {
|
||||
for i := 0; i < 3; i++ {
|
||||
fmt.Printf(" %.2x", r[i])
|
||||
}
|
||||
fmt.Printf(" ..")
|
||||
for i := len(r) - 3; i < len(r); i++ {
|
||||
fmt.Printf(" %.2x", r[i])
|
||||
}
|
||||
}
|
||||
fmt.Printf("\n")
|
||||
|
||||
case WireFixed32:
|
||||
u, err = p.DecodeFixed32()
|
||||
if err != nil {
|
||||
fmt.Printf("%3d: t=%3d fix32 err %v\n", index, tag, err)
|
||||
break out
|
||||
}
|
||||
fmt.Printf("%3d: t=%3d fix32 %d\n", index, tag, u)
|
||||
|
||||
case WireFixed64:
|
||||
u, err = p.DecodeFixed64()
|
||||
if err != nil {
|
||||
fmt.Printf("%3d: t=%3d fix64 err %v\n", index, tag, err)
|
||||
break out
|
||||
}
|
||||
fmt.Printf("%3d: t=%3d fix64 %d\n", index, tag, u)
|
||||
|
||||
case WireVarint:
|
||||
u, err = p.DecodeVarint()
|
||||
if err != nil {
|
||||
fmt.Printf("%3d: t=%3d varint err %v\n", index, tag, err)
|
||||
break out
|
||||
}
|
||||
fmt.Printf("%3d: t=%3d varint %d\n", index, tag, u)
|
||||
|
||||
case WireStartGroup:
|
||||
fmt.Printf("%3d: t=%3d start\n", index, tag)
|
||||
depth++
|
||||
|
||||
case WireEndGroup:
|
||||
depth--
|
||||
fmt.Printf("%3d: t=%3d end\n", index, tag)
|
||||
}
|
||||
}
|
||||
|
||||
if depth != 0 {
|
||||
fmt.Printf("%3d: start-end not balanced %d\n", p.index, depth)
|
||||
}
|
||||
fmt.Printf("\n")
|
||||
|
||||
p.buf = obuf
|
||||
p.index = index
|
||||
}
|
||||
|
||||
// SetDefaults sets unset protocol buffer fields to their default values.
|
||||
// It only modifies fields that are both unset and have defined defaults.
|
||||
// It recursively sets default values in any non-nil sub-messages.
|
||||
func SetDefaults(pb Message) {
|
||||
setDefaults(reflect.ValueOf(pb), true, false)
|
||||
}
|
||||
|
||||
// v is a pointer to a struct.
|
||||
func setDefaults(v reflect.Value, recur, zeros bool) {
|
||||
v = v.Elem()
|
||||
|
||||
defaultMu.RLock()
|
||||
dm, ok := defaults[v.Type()]
|
||||
defaultMu.RUnlock()
|
||||
if !ok {
|
||||
dm = buildDefaultMessage(v.Type())
|
||||
defaultMu.Lock()
|
||||
defaults[v.Type()] = dm
|
||||
defaultMu.Unlock()
|
||||
}
|
||||
|
||||
for _, sf := range dm.scalars {
|
||||
f := v.Field(sf.index)
|
||||
if !f.IsNil() {
|
||||
// field already set
|
||||
continue
|
||||
}
|
||||
dv := sf.value
|
||||
if dv == nil && !zeros {
|
||||
// no explicit default, and don't want to set zeros
|
||||
continue
|
||||
}
|
||||
fptr := f.Addr().Interface() // **T
|
||||
// TODO: Consider batching the allocations we do here.
|
||||
switch sf.kind {
|
||||
case reflect.Bool:
|
||||
b := new(bool)
|
||||
if dv != nil {
|
||||
*b = dv.(bool)
|
||||
}
|
||||
*(fptr.(**bool)) = b
|
||||
case reflect.Float32:
|
||||
f := new(float32)
|
||||
if dv != nil {
|
||||
*f = dv.(float32)
|
||||
}
|
||||
*(fptr.(**float32)) = f
|
||||
case reflect.Float64:
|
||||
f := new(float64)
|
||||
if dv != nil {
|
||||
*f = dv.(float64)
|
||||
}
|
||||
*(fptr.(**float64)) = f
|
||||
case reflect.Int32:
|
||||
// might be an enum
|
||||
if ft := f.Type(); ft != int32PtrType {
|
||||
// enum
|
||||
f.Set(reflect.New(ft.Elem()))
|
||||
if dv != nil {
|
||||
f.Elem().SetInt(int64(dv.(int32)))
|
||||
}
|
||||
} else {
|
||||
// int32 field
|
||||
i := new(int32)
|
||||
if dv != nil {
|
||||
*i = dv.(int32)
|
||||
}
|
||||
*(fptr.(**int32)) = i
|
||||
}
|
||||
case reflect.Int64:
|
||||
i := new(int64)
|
||||
if dv != nil {
|
||||
*i = dv.(int64)
|
||||
}
|
||||
*(fptr.(**int64)) = i
|
||||
case reflect.String:
|
||||
s := new(string)
|
||||
if dv != nil {
|
||||
*s = dv.(string)
|
||||
}
|
||||
*(fptr.(**string)) = s
|
||||
case reflect.Uint8:
|
||||
// exceptional case: []byte
|
||||
var b []byte
|
||||
if dv != nil {
|
||||
db := dv.([]byte)
|
||||
b = make([]byte, len(db))
|
||||
copy(b, db)
|
||||
} else {
|
||||
b = []byte{}
|
||||
}
|
||||
*(fptr.(*[]byte)) = b
|
||||
case reflect.Uint32:
|
||||
u := new(uint32)
|
||||
if dv != nil {
|
||||
*u = dv.(uint32)
|
||||
}
|
||||
*(fptr.(**uint32)) = u
|
||||
case reflect.Uint64:
|
||||
u := new(uint64)
|
||||
if dv != nil {
|
||||
*u = dv.(uint64)
|
||||
}
|
||||
*(fptr.(**uint64)) = u
|
||||
default:
|
||||
log.Printf("proto: can't set default for field %v (sf.kind=%v)", f, sf.kind)
|
||||
}
|
||||
}
|
||||
|
||||
for _, ni := range dm.nested {
|
||||
f := v.Field(ni)
|
||||
// f is *T or []*T or map[T]*T
|
||||
switch f.Kind() {
|
||||
case reflect.Ptr:
|
||||
if f.IsNil() {
|
||||
continue
|
||||
}
|
||||
setDefaults(f, recur, zeros)
|
||||
|
||||
case reflect.Slice:
|
||||
for i := 0; i < f.Len(); i++ {
|
||||
e := f.Index(i)
|
||||
if e.IsNil() {
|
||||
continue
|
||||
}
|
||||
setDefaults(e, recur, zeros)
|
||||
}
|
||||
|
||||
case reflect.Map:
|
||||
for _, k := range f.MapKeys() {
|
||||
e := f.MapIndex(k)
|
||||
if e.IsNil() {
|
||||
continue
|
||||
}
|
||||
setDefaults(e, recur, zeros)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var (
|
||||
// defaults maps a protocol buffer struct type to a slice of the fields,
|
||||
// with its scalar fields set to their proto-declared non-zero default values.
|
||||
defaultMu sync.RWMutex
|
||||
defaults = make(map[reflect.Type]defaultMessage)
|
||||
|
||||
int32PtrType = reflect.TypeOf((*int32)(nil))
|
||||
)
|
||||
|
||||
// defaultMessage represents information about the default values of a message.
|
||||
type defaultMessage struct {
|
||||
scalars []scalarField
|
||||
nested []int // struct field index of nested messages
|
||||
}
|
||||
|
||||
type scalarField struct {
|
||||
index int // struct field index
|
||||
kind reflect.Kind // element type (the T in *T or []T)
|
||||
value interface{} // the proto-declared default value, or nil
|
||||
}
|
||||
|
||||
// t is a struct type.
|
||||
func buildDefaultMessage(t reflect.Type) (dm defaultMessage) {
|
||||
sprop := GetProperties(t)
|
||||
for _, prop := range sprop.Prop {
|
||||
fi, ok := sprop.decoderTags.get(prop.Tag)
|
||||
if !ok {
|
||||
// XXX_unrecognized
|
||||
continue
|
||||
}
|
||||
ft := t.Field(fi).Type
|
||||
|
||||
sf, nested, err := fieldDefault(ft, prop)
|
||||
switch {
|
||||
case err != nil:
|
||||
log.Print(err)
|
||||
case nested:
|
||||
dm.nested = append(dm.nested, fi)
|
||||
case sf != nil:
|
||||
sf.index = fi
|
||||
dm.scalars = append(dm.scalars, *sf)
|
||||
}
|
||||
}
|
||||
|
||||
return dm
|
||||
}
|
||||
|
||||
// fieldDefault returns the scalarField for field type ft.
|
||||
// sf will be nil if the field can not have a default.
|
||||
// nestedMessage will be true if this is a nested message.
|
||||
// Note that sf.index is not set on return.
|
||||
func fieldDefault(ft reflect.Type, prop *Properties) (sf *scalarField, nestedMessage bool, err error) {
|
||||
var canHaveDefault bool
|
||||
switch ft.Kind() {
|
||||
case reflect.Ptr:
|
||||
if ft.Elem().Kind() == reflect.Struct {
|
||||
nestedMessage = true
|
||||
} else {
|
||||
canHaveDefault = true // proto2 scalar field
|
||||
}
|
||||
|
||||
case reflect.Slice:
|
||||
switch ft.Elem().Kind() {
|
||||
case reflect.Ptr:
|
||||
nestedMessage = true // repeated message
|
||||
case reflect.Uint8:
|
||||
canHaveDefault = true // bytes field
|
||||
}
|
||||
|
||||
case reflect.Map:
|
||||
if ft.Elem().Kind() == reflect.Ptr {
|
||||
nestedMessage = true // map with message values
|
||||
}
|
||||
}
|
||||
|
||||
if !canHaveDefault {
|
||||
if nestedMessage {
|
||||
return nil, true, nil
|
||||
}
|
||||
return nil, false, nil
|
||||
}
|
||||
|
||||
// We now know that ft is a pointer or slice.
|
||||
sf = &scalarField{kind: ft.Elem().Kind()}
|
||||
|
||||
// scalar fields without defaults
|
||||
if !prop.HasDefault {
|
||||
return sf, false, nil
|
||||
}
|
||||
|
||||
// a scalar field: either *T or []byte
|
||||
switch ft.Elem().Kind() {
|
||||
case reflect.Bool:
|
||||
x, err := strconv.ParseBool(prop.Default)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default bool %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = x
|
||||
case reflect.Float32:
|
||||
x, err := strconv.ParseFloat(prop.Default, 32)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default float32 %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = float32(x)
|
||||
case reflect.Float64:
|
||||
x, err := strconv.ParseFloat(prop.Default, 64)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default float64 %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = x
|
||||
case reflect.Int32:
|
||||
x, err := strconv.ParseInt(prop.Default, 10, 32)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default int32 %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = int32(x)
|
||||
case reflect.Int64:
|
||||
x, err := strconv.ParseInt(prop.Default, 10, 64)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default int64 %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = x
|
||||
case reflect.String:
|
||||
sf.value = prop.Default
|
||||
case reflect.Uint8:
|
||||
// []byte (not *uint8)
|
||||
sf.value = []byte(prop.Default)
|
||||
case reflect.Uint32:
|
||||
x, err := strconv.ParseUint(prop.Default, 10, 32)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default uint32 %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = uint32(x)
|
||||
case reflect.Uint64:
|
||||
x, err := strconv.ParseUint(prop.Default, 10, 64)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("proto: bad default uint64 %q: %v", prop.Default, err)
|
||||
}
|
||||
sf.value = x
|
||||
default:
|
||||
return nil, false, fmt.Errorf("proto: unhandled def kind %v", ft.Elem().Kind())
|
||||
}
|
||||
|
||||
return sf, false, nil
|
||||
}
|
||||
|
||||
// Map fields may have key types of non-float scalars, strings and enums.
|
||||
// The easiest way to sort them in some deterministic order is to use fmt.
|
||||
// If this turns out to be inefficient we can always consider other options,
|
||||
// such as doing a Schwartzian transform.
|
||||
|
||||
func mapKeys(vs []reflect.Value) sort.Interface {
|
||||
s := mapKeySorter{
|
||||
vs: vs,
|
||||
// default Less function: textual comparison
|
||||
less: func(a, b reflect.Value) bool {
|
||||
return fmt.Sprint(a.Interface()) < fmt.Sprint(b.Interface())
|
||||
},
|
||||
}
|
||||
|
||||
// Type specialization per https://developers.google.com/protocol-buffers/docs/proto#maps;
|
||||
// numeric keys are sorted numerically.
|
||||
if len(vs) == 0 {
|
||||
return s
|
||||
}
|
||||
switch vs[0].Kind() {
|
||||
case reflect.Int32, reflect.Int64:
|
||||
s.less = func(a, b reflect.Value) bool { return a.Int() < b.Int() }
|
||||
case reflect.Uint32, reflect.Uint64:
|
||||
s.less = func(a, b reflect.Value) bool { return a.Uint() < b.Uint() }
|
||||
}
|
||||
|
||||
return s
|
||||
}
|
||||
|
||||
type mapKeySorter struct {
|
||||
vs []reflect.Value
|
||||
less func(a, b reflect.Value) bool
|
||||
}
|
||||
|
||||
func (s mapKeySorter) Len() int { return len(s.vs) }
|
||||
func (s mapKeySorter) Swap(i, j int) { s.vs[i], s.vs[j] = s.vs[j], s.vs[i] }
|
||||
func (s mapKeySorter) Less(i, j int) bool {
|
||||
return s.less(s.vs[i], s.vs[j])
|
||||
}
|
||||
|
||||
// isProto3Zero reports whether v is a zero proto3 value.
|
||||
func isProto3Zero(v reflect.Value) bool {
|
||||
switch v.Kind() {
|
||||
case reflect.Bool:
|
||||
return !v.Bool()
|
||||
case reflect.Int32, reflect.Int64:
|
||||
return v.Int() == 0
|
||||
case reflect.Uint32, reflect.Uint64:
|
||||
return v.Uint() == 0
|
||||
case reflect.Float32, reflect.Float64:
|
||||
return v.Float() == 0
|
||||
case reflect.String:
|
||||
return v.String() == ""
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ProtoPackageIsVersion2 is referenced from generated protocol buffer files
|
||||
// to assert that that code is compatible with this version of the proto package.
|
||||
const ProtoPackageIsVersion2 = true
|
||||
|
||||
// ProtoPackageIsVersion1 is referenced from generated protocol buffer files
|
||||
// to assert that that code is compatible with this version of the proto package.
|
||||
const ProtoPackageIsVersion1 = true
|
||||
+311
@@ -0,0 +1,311 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
package proto
|
||||
|
||||
/*
|
||||
* Support for message sets.
|
||||
*/
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// errNoMessageTypeID occurs when a protocol buffer does not have a message type ID.
|
||||
// A message type ID is required for storing a protocol buffer in a message set.
|
||||
var errNoMessageTypeID = errors.New("proto does not have a message type ID")
|
||||
|
||||
// The first two types (_MessageSet_Item and messageSet)
|
||||
// model what the protocol compiler produces for the following protocol message:
|
||||
// message MessageSet {
|
||||
// repeated group Item = 1 {
|
||||
// required int32 type_id = 2;
|
||||
// required string message = 3;
|
||||
// };
|
||||
// }
|
||||
// That is the MessageSet wire format. We can't use a proto to generate these
|
||||
// because that would introduce a circular dependency between it and this package.
|
||||
|
||||
type _MessageSet_Item struct {
|
||||
TypeId *int32 `protobuf:"varint,2,req,name=type_id"`
|
||||
Message []byte `protobuf:"bytes,3,req,name=message"`
|
||||
}
|
||||
|
||||
type messageSet struct {
|
||||
Item []*_MessageSet_Item `protobuf:"group,1,rep"`
|
||||
XXX_unrecognized []byte
|
||||
// TODO: caching?
|
||||
}
|
||||
|
||||
// Make sure messageSet is a Message.
|
||||
var _ Message = (*messageSet)(nil)
|
||||
|
||||
// messageTypeIder is an interface satisfied by a protocol buffer type
|
||||
// that may be stored in a MessageSet.
|
||||
type messageTypeIder interface {
|
||||
MessageTypeId() int32
|
||||
}
|
||||
|
||||
func (ms *messageSet) find(pb Message) *_MessageSet_Item {
|
||||
mti, ok := pb.(messageTypeIder)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
id := mti.MessageTypeId()
|
||||
for _, item := range ms.Item {
|
||||
if *item.TypeId == id {
|
||||
return item
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (ms *messageSet) Has(pb Message) bool {
|
||||
if ms.find(pb) != nil {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (ms *messageSet) Unmarshal(pb Message) error {
|
||||
if item := ms.find(pb); item != nil {
|
||||
return Unmarshal(item.Message, pb)
|
||||
}
|
||||
if _, ok := pb.(messageTypeIder); !ok {
|
||||
return errNoMessageTypeID
|
||||
}
|
||||
return nil // TODO: return error instead?
|
||||
}
|
||||
|
||||
func (ms *messageSet) Marshal(pb Message) error {
|
||||
msg, err := Marshal(pb)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if item := ms.find(pb); item != nil {
|
||||
// reuse existing item
|
||||
item.Message = msg
|
||||
return nil
|
||||
}
|
||||
|
||||
mti, ok := pb.(messageTypeIder)
|
||||
if !ok {
|
||||
return errNoMessageTypeID
|
||||
}
|
||||
|
||||
mtid := mti.MessageTypeId()
|
||||
ms.Item = append(ms.Item, &_MessageSet_Item{
|
||||
TypeId: &mtid,
|
||||
Message: msg,
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (ms *messageSet) Reset() { *ms = messageSet{} }
|
||||
func (ms *messageSet) String() string { return CompactTextString(ms) }
|
||||
func (*messageSet) ProtoMessage() {}
|
||||
|
||||
// Support for the message_set_wire_format message option.
|
||||
|
||||
func skipVarint(buf []byte) []byte {
|
||||
i := 0
|
||||
for ; buf[i]&0x80 != 0; i++ {
|
||||
}
|
||||
return buf[i+1:]
|
||||
}
|
||||
|
||||
// MarshalMessageSet encodes the extension map represented by m in the message set wire format.
|
||||
// It is called by generated Marshal methods on protocol buffer messages with the message_set_wire_format option.
|
||||
func MarshalMessageSet(exts interface{}) ([]byte, error) {
|
||||
var m map[int32]Extension
|
||||
switch exts := exts.(type) {
|
||||
case *XXX_InternalExtensions:
|
||||
if err := encodeExtensions(exts); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m, _ = exts.extensionsRead()
|
||||
case map[int32]Extension:
|
||||
if err := encodeExtensionsMap(exts); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m = exts
|
||||
default:
|
||||
return nil, errors.New("proto: not an extension map")
|
||||
}
|
||||
|
||||
// Sort extension IDs to provide a deterministic encoding.
|
||||
// See also enc_map in encode.go.
|
||||
ids := make([]int, 0, len(m))
|
||||
for id := range m {
|
||||
ids = append(ids, int(id))
|
||||
}
|
||||
sort.Ints(ids)
|
||||
|
||||
ms := &messageSet{Item: make([]*_MessageSet_Item, 0, len(m))}
|
||||
for _, id := range ids {
|
||||
e := m[int32(id)]
|
||||
// Remove the wire type and field number varint, as well as the length varint.
|
||||
msg := skipVarint(skipVarint(e.enc))
|
||||
|
||||
ms.Item = append(ms.Item, &_MessageSet_Item{
|
||||
TypeId: Int32(int32(id)),
|
||||
Message: msg,
|
||||
})
|
||||
}
|
||||
return Marshal(ms)
|
||||
}
|
||||
|
||||
// UnmarshalMessageSet decodes the extension map encoded in buf in the message set wire format.
|
||||
// It is called by generated Unmarshal methods on protocol buffer messages with the message_set_wire_format option.
|
||||
func UnmarshalMessageSet(buf []byte, exts interface{}) error {
|
||||
var m map[int32]Extension
|
||||
switch exts := exts.(type) {
|
||||
case *XXX_InternalExtensions:
|
||||
m = exts.extensionsWrite()
|
||||
case map[int32]Extension:
|
||||
m = exts
|
||||
default:
|
||||
return errors.New("proto: not an extension map")
|
||||
}
|
||||
|
||||
ms := new(messageSet)
|
||||
if err := Unmarshal(buf, ms); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, item := range ms.Item {
|
||||
id := *item.TypeId
|
||||
msg := item.Message
|
||||
|
||||
// Restore wire type and field number varint, plus length varint.
|
||||
// Be careful to preserve duplicate items.
|
||||
b := EncodeVarint(uint64(id)<<3 | WireBytes)
|
||||
if ext, ok := m[id]; ok {
|
||||
// Existing data; rip off the tag and length varint
|
||||
// so we join the new data correctly.
|
||||
// We can assume that ext.enc is set because we are unmarshaling.
|
||||
o := ext.enc[len(b):] // skip wire type and field number
|
||||
_, n := DecodeVarint(o) // calculate length of length varint
|
||||
o = o[n:] // skip length varint
|
||||
msg = append(o, msg...) // join old data and new data
|
||||
}
|
||||
b = append(b, EncodeVarint(uint64(len(msg)))...)
|
||||
b = append(b, msg...)
|
||||
|
||||
m[id] = Extension{enc: b}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// MarshalMessageSetJSON encodes the extension map represented by m in JSON format.
|
||||
// It is called by generated MarshalJSON methods on protocol buffer messages with the message_set_wire_format option.
|
||||
func MarshalMessageSetJSON(exts interface{}) ([]byte, error) {
|
||||
var m map[int32]Extension
|
||||
switch exts := exts.(type) {
|
||||
case *XXX_InternalExtensions:
|
||||
m, _ = exts.extensionsRead()
|
||||
case map[int32]Extension:
|
||||
m = exts
|
||||
default:
|
||||
return nil, errors.New("proto: not an extension map")
|
||||
}
|
||||
var b bytes.Buffer
|
||||
b.WriteByte('{')
|
||||
|
||||
// Process the map in key order for deterministic output.
|
||||
ids := make([]int32, 0, len(m))
|
||||
for id := range m {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Sort(int32Slice(ids)) // int32Slice defined in text.go
|
||||
|
||||
for i, id := range ids {
|
||||
ext := m[id]
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
|
||||
msd, ok := messageSetMap[id]
|
||||
if !ok {
|
||||
// Unknown type; we can't render it, so skip it.
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, `"[%s]":`, msd.name)
|
||||
|
||||
x := ext.value
|
||||
if x == nil {
|
||||
x = reflect.New(msd.t.Elem()).Interface()
|
||||
if err := Unmarshal(ext.enc, x.(Message)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
d, err := json.Marshal(x)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.Write(d)
|
||||
}
|
||||
b.WriteByte('}')
|
||||
return b.Bytes(), nil
|
||||
}
|
||||
|
||||
// UnmarshalMessageSetJSON decodes the extension map encoded in buf in JSON format.
|
||||
// It is called by generated UnmarshalJSON methods on protocol buffer messages with the message_set_wire_format option.
|
||||
func UnmarshalMessageSetJSON(buf []byte, exts interface{}) error {
|
||||
// Common-case fast path.
|
||||
if len(buf) == 0 || bytes.Equal(buf, []byte("{}")) {
|
||||
return nil
|
||||
}
|
||||
|
||||
// This is fairly tricky, and it's not clear that it is needed.
|
||||
return errors.New("TODO: UnmarshalMessageSetJSON not yet implemented")
|
||||
}
|
||||
|
||||
// A global registry of types that can be used in a MessageSet.
|
||||
|
||||
var messageSetMap = make(map[int32]messageSetDesc)
|
||||
|
||||
type messageSetDesc struct {
|
||||
t reflect.Type // pointer to struct
|
||||
name string
|
||||
}
|
||||
|
||||
// RegisterMessageSetType is called from the generated code.
|
||||
func RegisterMessageSetType(m Message, fieldNum int32, name string) {
|
||||
messageSetMap[fieldNum] = messageSetDesc{
|
||||
t: reflect.TypeOf(m),
|
||||
name: name,
|
||||
}
|
||||
}
|
||||
+484
@@ -0,0 +1,484 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2012 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
// +build appengine js
|
||||
|
||||
// This file contains an implementation of proto field accesses using package reflect.
|
||||
// It is slower than the code in pointer_unsafe.go but it avoids package unsafe and can
|
||||
// be used on App Engine.
|
||||
|
||||
package proto
|
||||
|
||||
import (
|
||||
"math"
|
||||
"reflect"
|
||||
)
|
||||
|
||||
// A structPointer is a pointer to a struct.
|
||||
type structPointer struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
// toStructPointer returns a structPointer equivalent to the given reflect value.
|
||||
// The reflect value must itself be a pointer to a struct.
|
||||
func toStructPointer(v reflect.Value) structPointer {
|
||||
return structPointer{v}
|
||||
}
|
||||
|
||||
// IsNil reports whether p is nil.
|
||||
func structPointer_IsNil(p structPointer) bool {
|
||||
return p.v.IsNil()
|
||||
}
|
||||
|
||||
// Interface returns the struct pointer as an interface value.
|
||||
func structPointer_Interface(p structPointer, _ reflect.Type) interface{} {
|
||||
return p.v.Interface()
|
||||
}
|
||||
|
||||
// A field identifies a field in a struct, accessible from a structPointer.
|
||||
// In this implementation, a field is identified by the sequence of field indices
|
||||
// passed to reflect's FieldByIndex.
|
||||
type field []int
|
||||
|
||||
// toField returns a field equivalent to the given reflect field.
|
||||
func toField(f *reflect.StructField) field {
|
||||
return f.Index
|
||||
}
|
||||
|
||||
// invalidField is an invalid field identifier.
|
||||
var invalidField = field(nil)
|
||||
|
||||
// IsValid reports whether the field identifier is valid.
|
||||
func (f field) IsValid() bool { return f != nil }
|
||||
|
||||
// field returns the given field in the struct as a reflect value.
|
||||
func structPointer_field(p structPointer, f field) reflect.Value {
|
||||
// Special case: an extension map entry with a value of type T
|
||||
// passes a *T to the struct-handling code with a zero field,
|
||||
// expecting that it will be treated as equivalent to *struct{ X T },
|
||||
// which has the same memory layout. We have to handle that case
|
||||
// specially, because reflect will panic if we call FieldByIndex on a
|
||||
// non-struct.
|
||||
if f == nil {
|
||||
return p.v.Elem()
|
||||
}
|
||||
|
||||
return p.v.Elem().FieldByIndex(f)
|
||||
}
|
||||
|
||||
// ifield returns the given field in the struct as an interface value.
|
||||
func structPointer_ifield(p structPointer, f field) interface{} {
|
||||
return structPointer_field(p, f).Addr().Interface()
|
||||
}
|
||||
|
||||
// Bytes returns the address of a []byte field in the struct.
|
||||
func structPointer_Bytes(p structPointer, f field) *[]byte {
|
||||
return structPointer_ifield(p, f).(*[]byte)
|
||||
}
|
||||
|
||||
// BytesSlice returns the address of a [][]byte field in the struct.
|
||||
func structPointer_BytesSlice(p structPointer, f field) *[][]byte {
|
||||
return structPointer_ifield(p, f).(*[][]byte)
|
||||
}
|
||||
|
||||
// Bool returns the address of a *bool field in the struct.
|
||||
func structPointer_Bool(p structPointer, f field) **bool {
|
||||
return structPointer_ifield(p, f).(**bool)
|
||||
}
|
||||
|
||||
// BoolVal returns the address of a bool field in the struct.
|
||||
func structPointer_BoolVal(p structPointer, f field) *bool {
|
||||
return structPointer_ifield(p, f).(*bool)
|
||||
}
|
||||
|
||||
// BoolSlice returns the address of a []bool field in the struct.
|
||||
func structPointer_BoolSlice(p structPointer, f field) *[]bool {
|
||||
return structPointer_ifield(p, f).(*[]bool)
|
||||
}
|
||||
|
||||
// String returns the address of a *string field in the struct.
|
||||
func structPointer_String(p structPointer, f field) **string {
|
||||
return structPointer_ifield(p, f).(**string)
|
||||
}
|
||||
|
||||
// StringVal returns the address of a string field in the struct.
|
||||
func structPointer_StringVal(p structPointer, f field) *string {
|
||||
return structPointer_ifield(p, f).(*string)
|
||||
}
|
||||
|
||||
// StringSlice returns the address of a []string field in the struct.
|
||||
func structPointer_StringSlice(p structPointer, f field) *[]string {
|
||||
return structPointer_ifield(p, f).(*[]string)
|
||||
}
|
||||
|
||||
// Extensions returns the address of an extension map field in the struct.
|
||||
func structPointer_Extensions(p structPointer, f field) *XXX_InternalExtensions {
|
||||
return structPointer_ifield(p, f).(*XXX_InternalExtensions)
|
||||
}
|
||||
|
||||
// ExtMap returns the address of an extension map field in the struct.
|
||||
func structPointer_ExtMap(p structPointer, f field) *map[int32]Extension {
|
||||
return structPointer_ifield(p, f).(*map[int32]Extension)
|
||||
}
|
||||
|
||||
// NewAt returns the reflect.Value for a pointer to a field in the struct.
|
||||
func structPointer_NewAt(p structPointer, f field, typ reflect.Type) reflect.Value {
|
||||
return structPointer_field(p, f).Addr()
|
||||
}
|
||||
|
||||
// SetStructPointer writes a *struct field in the struct.
|
||||
func structPointer_SetStructPointer(p structPointer, f field, q structPointer) {
|
||||
structPointer_field(p, f).Set(q.v)
|
||||
}
|
||||
|
||||
// GetStructPointer reads a *struct field in the struct.
|
||||
func structPointer_GetStructPointer(p structPointer, f field) structPointer {
|
||||
return structPointer{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
// StructPointerSlice the address of a []*struct field in the struct.
|
||||
func structPointer_StructPointerSlice(p structPointer, f field) structPointerSlice {
|
||||
return structPointerSlice{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
// A structPointerSlice represents the address of a slice of pointers to structs
|
||||
// (themselves messages or groups). That is, v.Type() is *[]*struct{...}.
|
||||
type structPointerSlice struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
func (p structPointerSlice) Len() int { return p.v.Len() }
|
||||
func (p structPointerSlice) Index(i int) structPointer { return structPointer{p.v.Index(i)} }
|
||||
func (p structPointerSlice) Append(q structPointer) {
|
||||
p.v.Set(reflect.Append(p.v, q.v))
|
||||
}
|
||||
|
||||
var (
|
||||
int32Type = reflect.TypeOf(int32(0))
|
||||
uint32Type = reflect.TypeOf(uint32(0))
|
||||
float32Type = reflect.TypeOf(float32(0))
|
||||
int64Type = reflect.TypeOf(int64(0))
|
||||
uint64Type = reflect.TypeOf(uint64(0))
|
||||
float64Type = reflect.TypeOf(float64(0))
|
||||
)
|
||||
|
||||
// A word32 represents a field of type *int32, *uint32, *float32, or *enum.
|
||||
// That is, v.Type() is *int32, *uint32, *float32, or *enum and v is assignable.
|
||||
type word32 struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
// IsNil reports whether p is nil.
|
||||
func word32_IsNil(p word32) bool {
|
||||
return p.v.IsNil()
|
||||
}
|
||||
|
||||
// Set sets p to point at a newly allocated word with bits set to x.
|
||||
func word32_Set(p word32, o *Buffer, x uint32) {
|
||||
t := p.v.Type().Elem()
|
||||
switch t {
|
||||
case int32Type:
|
||||
if len(o.int32s) == 0 {
|
||||
o.int32s = make([]int32, uint32PoolSize)
|
||||
}
|
||||
o.int32s[0] = int32(x)
|
||||
p.v.Set(reflect.ValueOf(&o.int32s[0]))
|
||||
o.int32s = o.int32s[1:]
|
||||
return
|
||||
case uint32Type:
|
||||
if len(o.uint32s) == 0 {
|
||||
o.uint32s = make([]uint32, uint32PoolSize)
|
||||
}
|
||||
o.uint32s[0] = x
|
||||
p.v.Set(reflect.ValueOf(&o.uint32s[0]))
|
||||
o.uint32s = o.uint32s[1:]
|
||||
return
|
||||
case float32Type:
|
||||
if len(o.float32s) == 0 {
|
||||
o.float32s = make([]float32, uint32PoolSize)
|
||||
}
|
||||
o.float32s[0] = math.Float32frombits(x)
|
||||
p.v.Set(reflect.ValueOf(&o.float32s[0]))
|
||||
o.float32s = o.float32s[1:]
|
||||
return
|
||||
}
|
||||
|
||||
// must be enum
|
||||
p.v.Set(reflect.New(t))
|
||||
p.v.Elem().SetInt(int64(int32(x)))
|
||||
}
|
||||
|
||||
// Get gets the bits pointed at by p, as a uint32.
|
||||
func word32_Get(p word32) uint32 {
|
||||
elem := p.v.Elem()
|
||||
switch elem.Kind() {
|
||||
case reflect.Int32:
|
||||
return uint32(elem.Int())
|
||||
case reflect.Uint32:
|
||||
return uint32(elem.Uint())
|
||||
case reflect.Float32:
|
||||
return math.Float32bits(float32(elem.Float()))
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
// Word32 returns a reference to a *int32, *uint32, *float32, or *enum field in the struct.
|
||||
func structPointer_Word32(p structPointer, f field) word32 {
|
||||
return word32{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
// A word32Val represents a field of type int32, uint32, float32, or enum.
|
||||
// That is, v.Type() is int32, uint32, float32, or enum and v is assignable.
|
||||
type word32Val struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
// Set sets *p to x.
|
||||
func word32Val_Set(p word32Val, x uint32) {
|
||||
switch p.v.Type() {
|
||||
case int32Type:
|
||||
p.v.SetInt(int64(x))
|
||||
return
|
||||
case uint32Type:
|
||||
p.v.SetUint(uint64(x))
|
||||
return
|
||||
case float32Type:
|
||||
p.v.SetFloat(float64(math.Float32frombits(x)))
|
||||
return
|
||||
}
|
||||
|
||||
// must be enum
|
||||
p.v.SetInt(int64(int32(x)))
|
||||
}
|
||||
|
||||
// Get gets the bits pointed at by p, as a uint32.
|
||||
func word32Val_Get(p word32Val) uint32 {
|
||||
elem := p.v
|
||||
switch elem.Kind() {
|
||||
case reflect.Int32:
|
||||
return uint32(elem.Int())
|
||||
case reflect.Uint32:
|
||||
return uint32(elem.Uint())
|
||||
case reflect.Float32:
|
||||
return math.Float32bits(float32(elem.Float()))
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
// Word32Val returns a reference to a int32, uint32, float32, or enum field in the struct.
|
||||
func structPointer_Word32Val(p structPointer, f field) word32Val {
|
||||
return word32Val{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
// A word32Slice is a slice of 32-bit values.
|
||||
// That is, v.Type() is []int32, []uint32, []float32, or []enum.
|
||||
type word32Slice struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
func (p word32Slice) Append(x uint32) {
|
||||
n, m := p.v.Len(), p.v.Cap()
|
||||
if n < m {
|
||||
p.v.SetLen(n + 1)
|
||||
} else {
|
||||
t := p.v.Type().Elem()
|
||||
p.v.Set(reflect.Append(p.v, reflect.Zero(t)))
|
||||
}
|
||||
elem := p.v.Index(n)
|
||||
switch elem.Kind() {
|
||||
case reflect.Int32:
|
||||
elem.SetInt(int64(int32(x)))
|
||||
case reflect.Uint32:
|
||||
elem.SetUint(uint64(x))
|
||||
case reflect.Float32:
|
||||
elem.SetFloat(float64(math.Float32frombits(x)))
|
||||
}
|
||||
}
|
||||
|
||||
func (p word32Slice) Len() int {
|
||||
return p.v.Len()
|
||||
}
|
||||
|
||||
func (p word32Slice) Index(i int) uint32 {
|
||||
elem := p.v.Index(i)
|
||||
switch elem.Kind() {
|
||||
case reflect.Int32:
|
||||
return uint32(elem.Int())
|
||||
case reflect.Uint32:
|
||||
return uint32(elem.Uint())
|
||||
case reflect.Float32:
|
||||
return math.Float32bits(float32(elem.Float()))
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
// Word32Slice returns a reference to a []int32, []uint32, []float32, or []enum field in the struct.
|
||||
func structPointer_Word32Slice(p structPointer, f field) word32Slice {
|
||||
return word32Slice{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
// word64 is like word32 but for 64-bit values.
|
||||
type word64 struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
func word64_Set(p word64, o *Buffer, x uint64) {
|
||||
t := p.v.Type().Elem()
|
||||
switch t {
|
||||
case int64Type:
|
||||
if len(o.int64s) == 0 {
|
||||
o.int64s = make([]int64, uint64PoolSize)
|
||||
}
|
||||
o.int64s[0] = int64(x)
|
||||
p.v.Set(reflect.ValueOf(&o.int64s[0]))
|
||||
o.int64s = o.int64s[1:]
|
||||
return
|
||||
case uint64Type:
|
||||
if len(o.uint64s) == 0 {
|
||||
o.uint64s = make([]uint64, uint64PoolSize)
|
||||
}
|
||||
o.uint64s[0] = x
|
||||
p.v.Set(reflect.ValueOf(&o.uint64s[0]))
|
||||
o.uint64s = o.uint64s[1:]
|
||||
return
|
||||
case float64Type:
|
||||
if len(o.float64s) == 0 {
|
||||
o.float64s = make([]float64, uint64PoolSize)
|
||||
}
|
||||
o.float64s[0] = math.Float64frombits(x)
|
||||
p.v.Set(reflect.ValueOf(&o.float64s[0]))
|
||||
o.float64s = o.float64s[1:]
|
||||
return
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
func word64_IsNil(p word64) bool {
|
||||
return p.v.IsNil()
|
||||
}
|
||||
|
||||
func word64_Get(p word64) uint64 {
|
||||
elem := p.v.Elem()
|
||||
switch elem.Kind() {
|
||||
case reflect.Int64:
|
||||
return uint64(elem.Int())
|
||||
case reflect.Uint64:
|
||||
return elem.Uint()
|
||||
case reflect.Float64:
|
||||
return math.Float64bits(elem.Float())
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
func structPointer_Word64(p structPointer, f field) word64 {
|
||||
return word64{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
// word64Val is like word32Val but for 64-bit values.
|
||||
type word64Val struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
func word64Val_Set(p word64Val, o *Buffer, x uint64) {
|
||||
switch p.v.Type() {
|
||||
case int64Type:
|
||||
p.v.SetInt(int64(x))
|
||||
return
|
||||
case uint64Type:
|
||||
p.v.SetUint(x)
|
||||
return
|
||||
case float64Type:
|
||||
p.v.SetFloat(math.Float64frombits(x))
|
||||
return
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
func word64Val_Get(p word64Val) uint64 {
|
||||
elem := p.v
|
||||
switch elem.Kind() {
|
||||
case reflect.Int64:
|
||||
return uint64(elem.Int())
|
||||
case reflect.Uint64:
|
||||
return elem.Uint()
|
||||
case reflect.Float64:
|
||||
return math.Float64bits(elem.Float())
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
func structPointer_Word64Val(p structPointer, f field) word64Val {
|
||||
return word64Val{structPointer_field(p, f)}
|
||||
}
|
||||
|
||||
type word64Slice struct {
|
||||
v reflect.Value
|
||||
}
|
||||
|
||||
func (p word64Slice) Append(x uint64) {
|
||||
n, m := p.v.Len(), p.v.Cap()
|
||||
if n < m {
|
||||
p.v.SetLen(n + 1)
|
||||
} else {
|
||||
t := p.v.Type().Elem()
|
||||
p.v.Set(reflect.Append(p.v, reflect.Zero(t)))
|
||||
}
|
||||
elem := p.v.Index(n)
|
||||
switch elem.Kind() {
|
||||
case reflect.Int64:
|
||||
elem.SetInt(int64(int64(x)))
|
||||
case reflect.Uint64:
|
||||
elem.SetUint(uint64(x))
|
||||
case reflect.Float64:
|
||||
elem.SetFloat(float64(math.Float64frombits(x)))
|
||||
}
|
||||
}
|
||||
|
||||
func (p word64Slice) Len() int {
|
||||
return p.v.Len()
|
||||
}
|
||||
|
||||
func (p word64Slice) Index(i int) uint64 {
|
||||
elem := p.v.Index(i)
|
||||
switch elem.Kind() {
|
||||
case reflect.Int64:
|
||||
return uint64(elem.Int())
|
||||
case reflect.Uint64:
|
||||
return uint64(elem.Uint())
|
||||
case reflect.Float64:
|
||||
return math.Float64bits(float64(elem.Float()))
|
||||
}
|
||||
panic("unreachable")
|
||||
}
|
||||
|
||||
func structPointer_Word64Slice(p structPointer, f field) word64Slice {
|
||||
return word64Slice{structPointer_field(p, f)}
|
||||
}
|
||||
+270
@@ -0,0 +1,270 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2012 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
// +build !appengine,!js
|
||||
|
||||
// This file contains the implementation of the proto field accesses using package unsafe.
|
||||
|
||||
package proto
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
// NOTE: These type_Foo functions would more idiomatically be methods,
|
||||
// but Go does not allow methods on pointer types, and we must preserve
|
||||
// some pointer type for the garbage collector. We use these
|
||||
// funcs with clunky names as our poor approximation to methods.
|
||||
//
|
||||
// An alternative would be
|
||||
// type structPointer struct { p unsafe.Pointer }
|
||||
// but that does not registerize as well.
|
||||
|
||||
// A structPointer is a pointer to a struct.
|
||||
type structPointer unsafe.Pointer
|
||||
|
||||
// toStructPointer returns a structPointer equivalent to the given reflect value.
|
||||
func toStructPointer(v reflect.Value) structPointer {
|
||||
return structPointer(unsafe.Pointer(v.Pointer()))
|
||||
}
|
||||
|
||||
// IsNil reports whether p is nil.
|
||||
func structPointer_IsNil(p structPointer) bool {
|
||||
return p == nil
|
||||
}
|
||||
|
||||
// Interface returns the struct pointer, assumed to have element type t,
|
||||
// as an interface value.
|
||||
func structPointer_Interface(p structPointer, t reflect.Type) interface{} {
|
||||
return reflect.NewAt(t, unsafe.Pointer(p)).Interface()
|
||||
}
|
||||
|
||||
// A field identifies a field in a struct, accessible from a structPointer.
|
||||
// In this implementation, a field is identified by its byte offset from the start of the struct.
|
||||
type field uintptr
|
||||
|
||||
// toField returns a field equivalent to the given reflect field.
|
||||
func toField(f *reflect.StructField) field {
|
||||
return field(f.Offset)
|
||||
}
|
||||
|
||||
// invalidField is an invalid field identifier.
|
||||
const invalidField = ^field(0)
|
||||
|
||||
// IsValid reports whether the field identifier is valid.
|
||||
func (f field) IsValid() bool {
|
||||
return f != ^field(0)
|
||||
}
|
||||
|
||||
// Bytes returns the address of a []byte field in the struct.
|
||||
func structPointer_Bytes(p structPointer, f field) *[]byte {
|
||||
return (*[]byte)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// BytesSlice returns the address of a [][]byte field in the struct.
|
||||
func structPointer_BytesSlice(p structPointer, f field) *[][]byte {
|
||||
return (*[][]byte)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// Bool returns the address of a *bool field in the struct.
|
||||
func structPointer_Bool(p structPointer, f field) **bool {
|
||||
return (**bool)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// BoolVal returns the address of a bool field in the struct.
|
||||
func structPointer_BoolVal(p structPointer, f field) *bool {
|
||||
return (*bool)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// BoolSlice returns the address of a []bool field in the struct.
|
||||
func structPointer_BoolSlice(p structPointer, f field) *[]bool {
|
||||
return (*[]bool)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// String returns the address of a *string field in the struct.
|
||||
func structPointer_String(p structPointer, f field) **string {
|
||||
return (**string)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// StringVal returns the address of a string field in the struct.
|
||||
func structPointer_StringVal(p structPointer, f field) *string {
|
||||
return (*string)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// StringSlice returns the address of a []string field in the struct.
|
||||
func structPointer_StringSlice(p structPointer, f field) *[]string {
|
||||
return (*[]string)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// ExtMap returns the address of an extension map field in the struct.
|
||||
func structPointer_Extensions(p structPointer, f field) *XXX_InternalExtensions {
|
||||
return (*XXX_InternalExtensions)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
func structPointer_ExtMap(p structPointer, f field) *map[int32]Extension {
|
||||
return (*map[int32]Extension)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// NewAt returns the reflect.Value for a pointer to a field in the struct.
|
||||
func structPointer_NewAt(p structPointer, f field, typ reflect.Type) reflect.Value {
|
||||
return reflect.NewAt(typ, unsafe.Pointer(uintptr(p)+uintptr(f)))
|
||||
}
|
||||
|
||||
// SetStructPointer writes a *struct field in the struct.
|
||||
func structPointer_SetStructPointer(p structPointer, f field, q structPointer) {
|
||||
*(*structPointer)(unsafe.Pointer(uintptr(p) + uintptr(f))) = q
|
||||
}
|
||||
|
||||
// GetStructPointer reads a *struct field in the struct.
|
||||
func structPointer_GetStructPointer(p structPointer, f field) structPointer {
|
||||
return *(*structPointer)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// StructPointerSlice the address of a []*struct field in the struct.
|
||||
func structPointer_StructPointerSlice(p structPointer, f field) *structPointerSlice {
|
||||
return (*structPointerSlice)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// A structPointerSlice represents a slice of pointers to structs (themselves submessages or groups).
|
||||
type structPointerSlice []structPointer
|
||||
|
||||
func (v *structPointerSlice) Len() int { return len(*v) }
|
||||
func (v *structPointerSlice) Index(i int) structPointer { return (*v)[i] }
|
||||
func (v *structPointerSlice) Append(p structPointer) { *v = append(*v, p) }
|
||||
|
||||
// A word32 is the address of a "pointer to 32-bit value" field.
|
||||
type word32 **uint32
|
||||
|
||||
// IsNil reports whether *v is nil.
|
||||
func word32_IsNil(p word32) bool {
|
||||
return *p == nil
|
||||
}
|
||||
|
||||
// Set sets *v to point at a newly allocated word set to x.
|
||||
func word32_Set(p word32, o *Buffer, x uint32) {
|
||||
if len(o.uint32s) == 0 {
|
||||
o.uint32s = make([]uint32, uint32PoolSize)
|
||||
}
|
||||
o.uint32s[0] = x
|
||||
*p = &o.uint32s[0]
|
||||
o.uint32s = o.uint32s[1:]
|
||||
}
|
||||
|
||||
// Get gets the value pointed at by *v.
|
||||
func word32_Get(p word32) uint32 {
|
||||
return **p
|
||||
}
|
||||
|
||||
// Word32 returns the address of a *int32, *uint32, *float32, or *enum field in the struct.
|
||||
func structPointer_Word32(p structPointer, f field) word32 {
|
||||
return word32((**uint32)(unsafe.Pointer(uintptr(p) + uintptr(f))))
|
||||
}
|
||||
|
||||
// A word32Val is the address of a 32-bit value field.
|
||||
type word32Val *uint32
|
||||
|
||||
// Set sets *p to x.
|
||||
func word32Val_Set(p word32Val, x uint32) {
|
||||
*p = x
|
||||
}
|
||||
|
||||
// Get gets the value pointed at by p.
|
||||
func word32Val_Get(p word32Val) uint32 {
|
||||
return *p
|
||||
}
|
||||
|
||||
// Word32Val returns the address of a *int32, *uint32, *float32, or *enum field in the struct.
|
||||
func structPointer_Word32Val(p structPointer, f field) word32Val {
|
||||
return word32Val((*uint32)(unsafe.Pointer(uintptr(p) + uintptr(f))))
|
||||
}
|
||||
|
||||
// A word32Slice is a slice of 32-bit values.
|
||||
type word32Slice []uint32
|
||||
|
||||
func (v *word32Slice) Append(x uint32) { *v = append(*v, x) }
|
||||
func (v *word32Slice) Len() int { return len(*v) }
|
||||
func (v *word32Slice) Index(i int) uint32 { return (*v)[i] }
|
||||
|
||||
// Word32Slice returns the address of a []int32, []uint32, []float32, or []enum field in the struct.
|
||||
func structPointer_Word32Slice(p structPointer, f field) *word32Slice {
|
||||
return (*word32Slice)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
|
||||
// word64 is like word32 but for 64-bit values.
|
||||
type word64 **uint64
|
||||
|
||||
func word64_Set(p word64, o *Buffer, x uint64) {
|
||||
if len(o.uint64s) == 0 {
|
||||
o.uint64s = make([]uint64, uint64PoolSize)
|
||||
}
|
||||
o.uint64s[0] = x
|
||||
*p = &o.uint64s[0]
|
||||
o.uint64s = o.uint64s[1:]
|
||||
}
|
||||
|
||||
func word64_IsNil(p word64) bool {
|
||||
return *p == nil
|
||||
}
|
||||
|
||||
func word64_Get(p word64) uint64 {
|
||||
return **p
|
||||
}
|
||||
|
||||
func structPointer_Word64(p structPointer, f field) word64 {
|
||||
return word64((**uint64)(unsafe.Pointer(uintptr(p) + uintptr(f))))
|
||||
}
|
||||
|
||||
// word64Val is like word32Val but for 64-bit values.
|
||||
type word64Val *uint64
|
||||
|
||||
func word64Val_Set(p word64Val, o *Buffer, x uint64) {
|
||||
*p = x
|
||||
}
|
||||
|
||||
func word64Val_Get(p word64Val) uint64 {
|
||||
return *p
|
||||
}
|
||||
|
||||
func structPointer_Word64Val(p structPointer, f field) word64Val {
|
||||
return word64Val((*uint64)(unsafe.Pointer(uintptr(p) + uintptr(f))))
|
||||
}
|
||||
|
||||
// word64Slice is like word32Slice but for 64-bit values.
|
||||
type word64Slice []uint64
|
||||
|
||||
func (v *word64Slice) Append(x uint64) { *v = append(*v, x) }
|
||||
func (v *word64Slice) Len() int { return len(*v) }
|
||||
func (v *word64Slice) Index(i int) uint64 { return (*v)[i] }
|
||||
|
||||
func structPointer_Word64Slice(p structPointer, f field) *word64Slice {
|
||||
return (*word64Slice)(unsafe.Pointer(uintptr(p) + uintptr(f)))
|
||||
}
|
||||
+872
@@ -0,0 +1,872 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
package proto
|
||||
|
||||
/*
|
||||
* Routines for encoding data into the wire format for protocol buffers.
|
||||
*/
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
const debug bool = false
|
||||
|
||||
// Constants that identify the encoding of a value on the wire.
|
||||
const (
|
||||
WireVarint = 0
|
||||
WireFixed64 = 1
|
||||
WireBytes = 2
|
||||
WireStartGroup = 3
|
||||
WireEndGroup = 4
|
||||
WireFixed32 = 5
|
||||
)
|
||||
|
||||
const startSize = 10 // initial slice/string sizes
|
||||
|
||||
// Encoders are defined in encode.go
|
||||
// An encoder outputs the full representation of a field, including its
|
||||
// tag and encoder type.
|
||||
type encoder func(p *Buffer, prop *Properties, base structPointer) error
|
||||
|
||||
// A valueEncoder encodes a single integer in a particular encoding.
|
||||
type valueEncoder func(o *Buffer, x uint64) error
|
||||
|
||||
// Sizers are defined in encode.go
|
||||
// A sizer returns the encoded size of a field, including its tag and encoder
|
||||
// type.
|
||||
type sizer func(prop *Properties, base structPointer) int
|
||||
|
||||
// A valueSizer returns the encoded size of a single integer in a particular
|
||||
// encoding.
|
||||
type valueSizer func(x uint64) int
|
||||
|
||||
// Decoders are defined in decode.go
|
||||
// A decoder creates a value from its wire representation.
|
||||
// Unrecognized subelements are saved in unrec.
|
||||
type decoder func(p *Buffer, prop *Properties, base structPointer) error
|
||||
|
||||
// A valueDecoder decodes a single integer in a particular encoding.
|
||||
type valueDecoder func(o *Buffer) (x uint64, err error)
|
||||
|
||||
// A oneofMarshaler does the marshaling for all oneof fields in a message.
|
||||
type oneofMarshaler func(Message, *Buffer) error
|
||||
|
||||
// A oneofUnmarshaler does the unmarshaling for a oneof field in a message.
|
||||
type oneofUnmarshaler func(Message, int, int, *Buffer) (bool, error)
|
||||
|
||||
// A oneofSizer does the sizing for all oneof fields in a message.
|
||||
type oneofSizer func(Message) int
|
||||
|
||||
// tagMap is an optimization over map[int]int for typical protocol buffer
|
||||
// use-cases. Encoded protocol buffers are often in tag order with small tag
|
||||
// numbers.
|
||||
type tagMap struct {
|
||||
fastTags []int
|
||||
slowTags map[int]int
|
||||
}
|
||||
|
||||
// tagMapFastLimit is the upper bound on the tag number that will be stored in
|
||||
// the tagMap slice rather than its map.
|
||||
const tagMapFastLimit = 1024
|
||||
|
||||
func (p *tagMap) get(t int) (int, bool) {
|
||||
if t > 0 && t < tagMapFastLimit {
|
||||
if t >= len(p.fastTags) {
|
||||
return 0, false
|
||||
}
|
||||
fi := p.fastTags[t]
|
||||
return fi, fi >= 0
|
||||
}
|
||||
fi, ok := p.slowTags[t]
|
||||
return fi, ok
|
||||
}
|
||||
|
||||
func (p *tagMap) put(t int, fi int) {
|
||||
if t > 0 && t < tagMapFastLimit {
|
||||
for len(p.fastTags) < t+1 {
|
||||
p.fastTags = append(p.fastTags, -1)
|
||||
}
|
||||
p.fastTags[t] = fi
|
||||
return
|
||||
}
|
||||
if p.slowTags == nil {
|
||||
p.slowTags = make(map[int]int)
|
||||
}
|
||||
p.slowTags[t] = fi
|
||||
}
|
||||
|
||||
// StructProperties represents properties for all the fields of a struct.
|
||||
// decoderTags and decoderOrigNames should only be used by the decoder.
|
||||
type StructProperties struct {
|
||||
Prop []*Properties // properties for each field
|
||||
reqCount int // required count
|
||||
decoderTags tagMap // map from proto tag to struct field number
|
||||
decoderOrigNames map[string]int // map from original name to struct field number
|
||||
order []int // list of struct field numbers in tag order
|
||||
unrecField field // field id of the XXX_unrecognized []byte field
|
||||
extendable bool // is this an extendable proto
|
||||
|
||||
oneofMarshaler oneofMarshaler
|
||||
oneofUnmarshaler oneofUnmarshaler
|
||||
oneofSizer oneofSizer
|
||||
stype reflect.Type
|
||||
|
||||
// OneofTypes contains information about the oneof fields in this message.
|
||||
// It is keyed by the original name of a field.
|
||||
OneofTypes map[string]*OneofProperties
|
||||
}
|
||||
|
||||
// OneofProperties represents information about a specific field in a oneof.
|
||||
type OneofProperties struct {
|
||||
Type reflect.Type // pointer to generated struct type for this oneof field
|
||||
Field int // struct field number of the containing oneof in the message
|
||||
Prop *Properties
|
||||
}
|
||||
|
||||
// Implement the sorting interface so we can sort the fields in tag order, as recommended by the spec.
|
||||
// See encode.go, (*Buffer).enc_struct.
|
||||
|
||||
func (sp *StructProperties) Len() int { return len(sp.order) }
|
||||
func (sp *StructProperties) Less(i, j int) bool {
|
||||
return sp.Prop[sp.order[i]].Tag < sp.Prop[sp.order[j]].Tag
|
||||
}
|
||||
func (sp *StructProperties) Swap(i, j int) { sp.order[i], sp.order[j] = sp.order[j], sp.order[i] }
|
||||
|
||||
// Properties represents the protocol-specific behavior of a single struct field.
|
||||
type Properties struct {
|
||||
Name string // name of the field, for error messages
|
||||
OrigName string // original name before protocol compiler (always set)
|
||||
JSONName string // name to use for JSON; determined by protoc
|
||||
Wire string
|
||||
WireType int
|
||||
Tag int
|
||||
Required bool
|
||||
Optional bool
|
||||
Repeated bool
|
||||
Packed bool // relevant for repeated primitives only
|
||||
Enum string // set for enum types only
|
||||
proto3 bool // whether this is known to be a proto3 field; set for []byte only
|
||||
oneof bool // whether this is a oneof field
|
||||
|
||||
Default string // default value
|
||||
HasDefault bool // whether an explicit default was provided
|
||||
def_uint64 uint64
|
||||
|
||||
enc encoder
|
||||
valEnc valueEncoder // set for bool and numeric types only
|
||||
field field
|
||||
tagcode []byte // encoding of EncodeVarint((Tag<<3)|WireType)
|
||||
tagbuf [8]byte
|
||||
stype reflect.Type // set for struct types only
|
||||
sprop *StructProperties // set for struct types only
|
||||
isMarshaler bool
|
||||
isUnmarshaler bool
|
||||
|
||||
mtype reflect.Type // set for map types only
|
||||
mkeyprop *Properties // set for map types only
|
||||
mvalprop *Properties // set for map types only
|
||||
|
||||
size sizer
|
||||
valSize valueSizer // set for bool and numeric types only
|
||||
|
||||
dec decoder
|
||||
valDec valueDecoder // set for bool and numeric types only
|
||||
|
||||
// If this is a packable field, this will be the decoder for the packed version of the field.
|
||||
packedDec decoder
|
||||
}
|
||||
|
||||
// String formats the properties in the protobuf struct field tag style.
|
||||
func (p *Properties) String() string {
|
||||
s := p.Wire
|
||||
s = ","
|
||||
s += strconv.Itoa(p.Tag)
|
||||
if p.Required {
|
||||
s += ",req"
|
||||
}
|
||||
if p.Optional {
|
||||
s += ",opt"
|
||||
}
|
||||
if p.Repeated {
|
||||
s += ",rep"
|
||||
}
|
||||
if p.Packed {
|
||||
s += ",packed"
|
||||
}
|
||||
s += ",name=" + p.OrigName
|
||||
if p.JSONName != p.OrigName {
|
||||
s += ",json=" + p.JSONName
|
||||
}
|
||||
if p.proto3 {
|
||||
s += ",proto3"
|
||||
}
|
||||
if p.oneof {
|
||||
s += ",oneof"
|
||||
}
|
||||
if len(p.Enum) > 0 {
|
||||
s += ",enum=" + p.Enum
|
||||
}
|
||||
if p.HasDefault {
|
||||
s += ",def=" + p.Default
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// Parse populates p by parsing a string in the protobuf struct field tag style.
|
||||
func (p *Properties) Parse(s string) {
|
||||
// "bytes,49,opt,name=foo,def=hello!"
|
||||
fields := strings.Split(s, ",") // breaks def=, but handled below.
|
||||
if len(fields) < 2 {
|
||||
fmt.Fprintf(os.Stderr, "proto: tag has too few fields: %q\n", s)
|
||||
return
|
||||
}
|
||||
|
||||
p.Wire = fields[0]
|
||||
switch p.Wire {
|
||||
case "varint":
|
||||
p.WireType = WireVarint
|
||||
p.valEnc = (*Buffer).EncodeVarint
|
||||
p.valDec = (*Buffer).DecodeVarint
|
||||
p.valSize = sizeVarint
|
||||
case "fixed32":
|
||||
p.WireType = WireFixed32
|
||||
p.valEnc = (*Buffer).EncodeFixed32
|
||||
p.valDec = (*Buffer).DecodeFixed32
|
||||
p.valSize = sizeFixed32
|
||||
case "fixed64":
|
||||
p.WireType = WireFixed64
|
||||
p.valEnc = (*Buffer).EncodeFixed64
|
||||
p.valDec = (*Buffer).DecodeFixed64
|
||||
p.valSize = sizeFixed64
|
||||
case "zigzag32":
|
||||
p.WireType = WireVarint
|
||||
p.valEnc = (*Buffer).EncodeZigzag32
|
||||
p.valDec = (*Buffer).DecodeZigzag32
|
||||
p.valSize = sizeZigzag32
|
||||
case "zigzag64":
|
||||
p.WireType = WireVarint
|
||||
p.valEnc = (*Buffer).EncodeZigzag64
|
||||
p.valDec = (*Buffer).DecodeZigzag64
|
||||
p.valSize = sizeZigzag64
|
||||
case "bytes", "group":
|
||||
p.WireType = WireBytes
|
||||
// no numeric converter for non-numeric types
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "proto: tag has unknown wire type: %q\n", s)
|
||||
return
|
||||
}
|
||||
|
||||
var err error
|
||||
p.Tag, err = strconv.Atoi(fields[1])
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
for i := 2; i < len(fields); i++ {
|
||||
f := fields[i]
|
||||
switch {
|
||||
case f == "req":
|
||||
p.Required = true
|
||||
case f == "opt":
|
||||
p.Optional = true
|
||||
case f == "rep":
|
||||
p.Repeated = true
|
||||
case f == "packed":
|
||||
p.Packed = true
|
||||
case strings.HasPrefix(f, "name="):
|
||||
p.OrigName = f[5:]
|
||||
case strings.HasPrefix(f, "json="):
|
||||
p.JSONName = f[5:]
|
||||
case strings.HasPrefix(f, "enum="):
|
||||
p.Enum = f[5:]
|
||||
case f == "proto3":
|
||||
p.proto3 = true
|
||||
case f == "oneof":
|
||||
p.oneof = true
|
||||
case strings.HasPrefix(f, "def="):
|
||||
p.HasDefault = true
|
||||
p.Default = f[4:] // rest of string
|
||||
if i+1 < len(fields) {
|
||||
// Commas aren't escaped, and def is always last.
|
||||
p.Default += "," + strings.Join(fields[i+1:], ",")
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func logNoSliceEnc(t1, t2 reflect.Type) {
|
||||
fmt.Fprintf(os.Stderr, "proto: no slice oenc for %T = []%T\n", t1, t2)
|
||||
}
|
||||
|
||||
var protoMessageType = reflect.TypeOf((*Message)(nil)).Elem()
|
||||
|
||||
// Initialize the fields for encoding and decoding.
|
||||
func (p *Properties) setEncAndDec(typ reflect.Type, f *reflect.StructField, lockGetProp bool) {
|
||||
p.enc = nil
|
||||
p.dec = nil
|
||||
p.size = nil
|
||||
|
||||
switch t1 := typ; t1.Kind() {
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "proto: no coders for %v\n", t1)
|
||||
|
||||
// proto3 scalar types
|
||||
|
||||
case reflect.Bool:
|
||||
p.enc = (*Buffer).enc_proto3_bool
|
||||
p.dec = (*Buffer).dec_proto3_bool
|
||||
p.size = size_proto3_bool
|
||||
case reflect.Int32:
|
||||
p.enc = (*Buffer).enc_proto3_int32
|
||||
p.dec = (*Buffer).dec_proto3_int32
|
||||
p.size = size_proto3_int32
|
||||
case reflect.Uint32:
|
||||
p.enc = (*Buffer).enc_proto3_uint32
|
||||
p.dec = (*Buffer).dec_proto3_int32 // can reuse
|
||||
p.size = size_proto3_uint32
|
||||
case reflect.Int64, reflect.Uint64:
|
||||
p.enc = (*Buffer).enc_proto3_int64
|
||||
p.dec = (*Buffer).dec_proto3_int64
|
||||
p.size = size_proto3_int64
|
||||
case reflect.Float32:
|
||||
p.enc = (*Buffer).enc_proto3_uint32 // can just treat them as bits
|
||||
p.dec = (*Buffer).dec_proto3_int32
|
||||
p.size = size_proto3_uint32
|
||||
case reflect.Float64:
|
||||
p.enc = (*Buffer).enc_proto3_int64 // can just treat them as bits
|
||||
p.dec = (*Buffer).dec_proto3_int64
|
||||
p.size = size_proto3_int64
|
||||
case reflect.String:
|
||||
p.enc = (*Buffer).enc_proto3_string
|
||||
p.dec = (*Buffer).dec_proto3_string
|
||||
p.size = size_proto3_string
|
||||
|
||||
case reflect.Ptr:
|
||||
switch t2 := t1.Elem(); t2.Kind() {
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "proto: no encoder function for %v -> %v\n", t1, t2)
|
||||
break
|
||||
case reflect.Bool:
|
||||
p.enc = (*Buffer).enc_bool
|
||||
p.dec = (*Buffer).dec_bool
|
||||
p.size = size_bool
|
||||
case reflect.Int32:
|
||||
p.enc = (*Buffer).enc_int32
|
||||
p.dec = (*Buffer).dec_int32
|
||||
p.size = size_int32
|
||||
case reflect.Uint32:
|
||||
p.enc = (*Buffer).enc_uint32
|
||||
p.dec = (*Buffer).dec_int32 // can reuse
|
||||
p.size = size_uint32
|
||||
case reflect.Int64, reflect.Uint64:
|
||||
p.enc = (*Buffer).enc_int64
|
||||
p.dec = (*Buffer).dec_int64
|
||||
p.size = size_int64
|
||||
case reflect.Float32:
|
||||
p.enc = (*Buffer).enc_uint32 // can just treat them as bits
|
||||
p.dec = (*Buffer).dec_int32
|
||||
p.size = size_uint32
|
||||
case reflect.Float64:
|
||||
p.enc = (*Buffer).enc_int64 // can just treat them as bits
|
||||
p.dec = (*Buffer).dec_int64
|
||||
p.size = size_int64
|
||||
case reflect.String:
|
||||
p.enc = (*Buffer).enc_string
|
||||
p.dec = (*Buffer).dec_string
|
||||
p.size = size_string
|
||||
case reflect.Struct:
|
||||
p.stype = t1.Elem()
|
||||
p.isMarshaler = isMarshaler(t1)
|
||||
p.isUnmarshaler = isUnmarshaler(t1)
|
||||
if p.Wire == "bytes" {
|
||||
p.enc = (*Buffer).enc_struct_message
|
||||
p.dec = (*Buffer).dec_struct_message
|
||||
p.size = size_struct_message
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_struct_group
|
||||
p.dec = (*Buffer).dec_struct_group
|
||||
p.size = size_struct_group
|
||||
}
|
||||
}
|
||||
|
||||
case reflect.Slice:
|
||||
switch t2 := t1.Elem(); t2.Kind() {
|
||||
default:
|
||||
logNoSliceEnc(t1, t2)
|
||||
break
|
||||
case reflect.Bool:
|
||||
if p.Packed {
|
||||
p.enc = (*Buffer).enc_slice_packed_bool
|
||||
p.size = size_slice_packed_bool
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_bool
|
||||
p.size = size_slice_bool
|
||||
}
|
||||
p.dec = (*Buffer).dec_slice_bool
|
||||
p.packedDec = (*Buffer).dec_slice_packed_bool
|
||||
case reflect.Int32:
|
||||
if p.Packed {
|
||||
p.enc = (*Buffer).enc_slice_packed_int32
|
||||
p.size = size_slice_packed_int32
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_int32
|
||||
p.size = size_slice_int32
|
||||
}
|
||||
p.dec = (*Buffer).dec_slice_int32
|
||||
p.packedDec = (*Buffer).dec_slice_packed_int32
|
||||
case reflect.Uint32:
|
||||
if p.Packed {
|
||||
p.enc = (*Buffer).enc_slice_packed_uint32
|
||||
p.size = size_slice_packed_uint32
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_uint32
|
||||
p.size = size_slice_uint32
|
||||
}
|
||||
p.dec = (*Buffer).dec_slice_int32
|
||||
p.packedDec = (*Buffer).dec_slice_packed_int32
|
||||
case reflect.Int64, reflect.Uint64:
|
||||
if p.Packed {
|
||||
p.enc = (*Buffer).enc_slice_packed_int64
|
||||
p.size = size_slice_packed_int64
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_int64
|
||||
p.size = size_slice_int64
|
||||
}
|
||||
p.dec = (*Buffer).dec_slice_int64
|
||||
p.packedDec = (*Buffer).dec_slice_packed_int64
|
||||
case reflect.Uint8:
|
||||
p.dec = (*Buffer).dec_slice_byte
|
||||
if p.proto3 {
|
||||
p.enc = (*Buffer).enc_proto3_slice_byte
|
||||
p.size = size_proto3_slice_byte
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_byte
|
||||
p.size = size_slice_byte
|
||||
}
|
||||
case reflect.Float32, reflect.Float64:
|
||||
switch t2.Bits() {
|
||||
case 32:
|
||||
// can just treat them as bits
|
||||
if p.Packed {
|
||||
p.enc = (*Buffer).enc_slice_packed_uint32
|
||||
p.size = size_slice_packed_uint32
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_uint32
|
||||
p.size = size_slice_uint32
|
||||
}
|
||||
p.dec = (*Buffer).dec_slice_int32
|
||||
p.packedDec = (*Buffer).dec_slice_packed_int32
|
||||
case 64:
|
||||
// can just treat them as bits
|
||||
if p.Packed {
|
||||
p.enc = (*Buffer).enc_slice_packed_int64
|
||||
p.size = size_slice_packed_int64
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_int64
|
||||
p.size = size_slice_int64
|
||||
}
|
||||
p.dec = (*Buffer).dec_slice_int64
|
||||
p.packedDec = (*Buffer).dec_slice_packed_int64
|
||||
default:
|
||||
logNoSliceEnc(t1, t2)
|
||||
break
|
||||
}
|
||||
case reflect.String:
|
||||
p.enc = (*Buffer).enc_slice_string
|
||||
p.dec = (*Buffer).dec_slice_string
|
||||
p.size = size_slice_string
|
||||
case reflect.Ptr:
|
||||
switch t3 := t2.Elem(); t3.Kind() {
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "proto: no ptr oenc for %T -> %T -> %T\n", t1, t2, t3)
|
||||
break
|
||||
case reflect.Struct:
|
||||
p.stype = t2.Elem()
|
||||
p.isMarshaler = isMarshaler(t2)
|
||||
p.isUnmarshaler = isUnmarshaler(t2)
|
||||
if p.Wire == "bytes" {
|
||||
p.enc = (*Buffer).enc_slice_struct_message
|
||||
p.dec = (*Buffer).dec_slice_struct_message
|
||||
p.size = size_slice_struct_message
|
||||
} else {
|
||||
p.enc = (*Buffer).enc_slice_struct_group
|
||||
p.dec = (*Buffer).dec_slice_struct_group
|
||||
p.size = size_slice_struct_group
|
||||
}
|
||||
}
|
||||
case reflect.Slice:
|
||||
switch t2.Elem().Kind() {
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "proto: no slice elem oenc for %T -> %T -> %T\n", t1, t2, t2.Elem())
|
||||
break
|
||||
case reflect.Uint8:
|
||||
p.enc = (*Buffer).enc_slice_slice_byte
|
||||
p.dec = (*Buffer).dec_slice_slice_byte
|
||||
p.size = size_slice_slice_byte
|
||||
}
|
||||
}
|
||||
|
||||
case reflect.Map:
|
||||
p.enc = (*Buffer).enc_new_map
|
||||
p.dec = (*Buffer).dec_new_map
|
||||
p.size = size_new_map
|
||||
|
||||
p.mtype = t1
|
||||
p.mkeyprop = &Properties{}
|
||||
p.mkeyprop.init(reflect.PtrTo(p.mtype.Key()), "Key", f.Tag.Get("protobuf_key"), nil, lockGetProp)
|
||||
p.mvalprop = &Properties{}
|
||||
vtype := p.mtype.Elem()
|
||||
if vtype.Kind() != reflect.Ptr && vtype.Kind() != reflect.Slice {
|
||||
// The value type is not a message (*T) or bytes ([]byte),
|
||||
// so we need encoders for the pointer to this type.
|
||||
vtype = reflect.PtrTo(vtype)
|
||||
}
|
||||
p.mvalprop.init(vtype, "Value", f.Tag.Get("protobuf_val"), nil, lockGetProp)
|
||||
}
|
||||
|
||||
// precalculate tag code
|
||||
wire := p.WireType
|
||||
if p.Packed {
|
||||
wire = WireBytes
|
||||
}
|
||||
x := uint32(p.Tag)<<3 | uint32(wire)
|
||||
i := 0
|
||||
for i = 0; x > 127; i++ {
|
||||
p.tagbuf[i] = 0x80 | uint8(x&0x7F)
|
||||
x >>= 7
|
||||
}
|
||||
p.tagbuf[i] = uint8(x)
|
||||
p.tagcode = p.tagbuf[0 : i+1]
|
||||
|
||||
if p.stype != nil {
|
||||
if lockGetProp {
|
||||
p.sprop = GetProperties(p.stype)
|
||||
} else {
|
||||
p.sprop = getPropertiesLocked(p.stype)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var (
|
||||
marshalerType = reflect.TypeOf((*Marshaler)(nil)).Elem()
|
||||
unmarshalerType = reflect.TypeOf((*Unmarshaler)(nil)).Elem()
|
||||
)
|
||||
|
||||
// isMarshaler reports whether type t implements Marshaler.
|
||||
func isMarshaler(t reflect.Type) bool {
|
||||
// We're checking for (likely) pointer-receiver methods
|
||||
// so if t is not a pointer, something is very wrong.
|
||||
// The calls above only invoke isMarshaler on pointer types.
|
||||
if t.Kind() != reflect.Ptr {
|
||||
panic("proto: misuse of isMarshaler")
|
||||
}
|
||||
return t.Implements(marshalerType)
|
||||
}
|
||||
|
||||
// isUnmarshaler reports whether type t implements Unmarshaler.
|
||||
func isUnmarshaler(t reflect.Type) bool {
|
||||
// We're checking for (likely) pointer-receiver methods
|
||||
// so if t is not a pointer, something is very wrong.
|
||||
// The calls above only invoke isUnmarshaler on pointer types.
|
||||
if t.Kind() != reflect.Ptr {
|
||||
panic("proto: misuse of isUnmarshaler")
|
||||
}
|
||||
return t.Implements(unmarshalerType)
|
||||
}
|
||||
|
||||
// Init populates the properties from a protocol buffer struct tag.
|
||||
func (p *Properties) Init(typ reflect.Type, name, tag string, f *reflect.StructField) {
|
||||
p.init(typ, name, tag, f, true)
|
||||
}
|
||||
|
||||
func (p *Properties) init(typ reflect.Type, name, tag string, f *reflect.StructField, lockGetProp bool) {
|
||||
// "bytes,49,opt,def=hello!"
|
||||
p.Name = name
|
||||
p.OrigName = name
|
||||
if f != nil {
|
||||
p.field = toField(f)
|
||||
}
|
||||
if tag == "" {
|
||||
return
|
||||
}
|
||||
p.Parse(tag)
|
||||
p.setEncAndDec(typ, f, lockGetProp)
|
||||
}
|
||||
|
||||
var (
|
||||
propertiesMu sync.RWMutex
|
||||
propertiesMap = make(map[reflect.Type]*StructProperties)
|
||||
)
|
||||
|
||||
// GetProperties returns the list of properties for the type represented by t.
|
||||
// t must represent a generated struct type of a protocol message.
|
||||
func GetProperties(t reflect.Type) *StructProperties {
|
||||
if t.Kind() != reflect.Struct {
|
||||
panic("proto: type must have kind struct")
|
||||
}
|
||||
|
||||
// Most calls to GetProperties in a long-running program will be
|
||||
// retrieving details for types we have seen before.
|
||||
propertiesMu.RLock()
|
||||
sprop, ok := propertiesMap[t]
|
||||
propertiesMu.RUnlock()
|
||||
if ok {
|
||||
if collectStats {
|
||||
stats.Chit++
|
||||
}
|
||||
return sprop
|
||||
}
|
||||
|
||||
propertiesMu.Lock()
|
||||
sprop = getPropertiesLocked(t)
|
||||
propertiesMu.Unlock()
|
||||
return sprop
|
||||
}
|
||||
|
||||
// getPropertiesLocked requires that propertiesMu is held.
|
||||
func getPropertiesLocked(t reflect.Type) *StructProperties {
|
||||
if prop, ok := propertiesMap[t]; ok {
|
||||
if collectStats {
|
||||
stats.Chit++
|
||||
}
|
||||
return prop
|
||||
}
|
||||
if collectStats {
|
||||
stats.Cmiss++
|
||||
}
|
||||
|
||||
prop := new(StructProperties)
|
||||
// in case of recursive protos, fill this in now.
|
||||
propertiesMap[t] = prop
|
||||
|
||||
// build properties
|
||||
prop.extendable = reflect.PtrTo(t).Implements(extendableProtoType) ||
|
||||
reflect.PtrTo(t).Implements(extendableProtoV1Type)
|
||||
prop.unrecField = invalidField
|
||||
prop.Prop = make([]*Properties, t.NumField())
|
||||
prop.order = make([]int, t.NumField())
|
||||
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
f := t.Field(i)
|
||||
p := new(Properties)
|
||||
name := f.Name
|
||||
p.init(f.Type, name, f.Tag.Get("protobuf"), &f, false)
|
||||
|
||||
if f.Name == "XXX_InternalExtensions" { // special case
|
||||
p.enc = (*Buffer).enc_exts
|
||||
p.dec = nil // not needed
|
||||
p.size = size_exts
|
||||
} else if f.Name == "XXX_extensions" { // special case
|
||||
p.enc = (*Buffer).enc_map
|
||||
p.dec = nil // not needed
|
||||
p.size = size_map
|
||||
} else if f.Name == "XXX_unrecognized" { // special case
|
||||
prop.unrecField = toField(&f)
|
||||
}
|
||||
oneof := f.Tag.Get("protobuf_oneof") // special case
|
||||
if oneof != "" {
|
||||
// Oneof fields don't use the traditional protobuf tag.
|
||||
p.OrigName = oneof
|
||||
}
|
||||
prop.Prop[i] = p
|
||||
prop.order[i] = i
|
||||
if debug {
|
||||
print(i, " ", f.Name, " ", t.String(), " ")
|
||||
if p.Tag > 0 {
|
||||
print(p.String())
|
||||
}
|
||||
print("\n")
|
||||
}
|
||||
if p.enc == nil && !strings.HasPrefix(f.Name, "XXX_") && oneof == "" {
|
||||
fmt.Fprintln(os.Stderr, "proto: no encoder for", f.Name, f.Type.String(), "[GetProperties]")
|
||||
}
|
||||
}
|
||||
|
||||
// Re-order prop.order.
|
||||
sort.Sort(prop)
|
||||
|
||||
type oneofMessage interface {
|
||||
XXX_OneofFuncs() (func(Message, *Buffer) error, func(Message, int, int, *Buffer) (bool, error), func(Message) int, []interface{})
|
||||
}
|
||||
if om, ok := reflect.Zero(reflect.PtrTo(t)).Interface().(oneofMessage); ok {
|
||||
var oots []interface{}
|
||||
prop.oneofMarshaler, prop.oneofUnmarshaler, prop.oneofSizer, oots = om.XXX_OneofFuncs()
|
||||
prop.stype = t
|
||||
|
||||
// Interpret oneof metadata.
|
||||
prop.OneofTypes = make(map[string]*OneofProperties)
|
||||
for _, oot := range oots {
|
||||
oop := &OneofProperties{
|
||||
Type: reflect.ValueOf(oot).Type(), // *T
|
||||
Prop: new(Properties),
|
||||
}
|
||||
sft := oop.Type.Elem().Field(0)
|
||||
oop.Prop.Name = sft.Name
|
||||
oop.Prop.Parse(sft.Tag.Get("protobuf"))
|
||||
// There will be exactly one interface field that
|
||||
// this new value is assignable to.
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
f := t.Field(i)
|
||||
if f.Type.Kind() != reflect.Interface {
|
||||
continue
|
||||
}
|
||||
if !oop.Type.AssignableTo(f.Type) {
|
||||
continue
|
||||
}
|
||||
oop.Field = i
|
||||
break
|
||||
}
|
||||
prop.OneofTypes[oop.Prop.OrigName] = oop
|
||||
}
|
||||
}
|
||||
|
||||
// build required counts
|
||||
// build tags
|
||||
reqCount := 0
|
||||
prop.decoderOrigNames = make(map[string]int)
|
||||
for i, p := range prop.Prop {
|
||||
if strings.HasPrefix(p.Name, "XXX_") {
|
||||
// Internal fields should not appear in tags/origNames maps.
|
||||
// They are handled specially when encoding and decoding.
|
||||
continue
|
||||
}
|
||||
if p.Required {
|
||||
reqCount++
|
||||
}
|
||||
prop.decoderTags.put(p.Tag, i)
|
||||
prop.decoderOrigNames[p.OrigName] = i
|
||||
}
|
||||
prop.reqCount = reqCount
|
||||
|
||||
return prop
|
||||
}
|
||||
|
||||
// Return the Properties object for the x[0]'th field of the structure.
|
||||
func propByIndex(t reflect.Type, x []int) *Properties {
|
||||
if len(x) != 1 {
|
||||
fmt.Fprintf(os.Stderr, "proto: field index dimension %d (not 1) for type %s\n", len(x), t)
|
||||
return nil
|
||||
}
|
||||
prop := GetProperties(t)
|
||||
return prop.Prop[x[0]]
|
||||
}
|
||||
|
||||
// Get the address and type of a pointer to a struct from an interface.
|
||||
func getbase(pb Message) (t reflect.Type, b structPointer, err error) {
|
||||
if pb == nil {
|
||||
err = ErrNil
|
||||
return
|
||||
}
|
||||
// get the reflect type of the pointer to the struct.
|
||||
t = reflect.TypeOf(pb)
|
||||
// get the address of the struct.
|
||||
value := reflect.ValueOf(pb)
|
||||
b = toStructPointer(value)
|
||||
return
|
||||
}
|
||||
|
||||
// A global registry of enum types.
|
||||
// The generated code will register the generated maps by calling RegisterEnum.
|
||||
|
||||
var enumValueMaps = make(map[string]map[string]int32)
|
||||
|
||||
// RegisterEnum is called from the generated code to install the enum descriptor
|
||||
// maps into the global table to aid parsing text format protocol buffers.
|
||||
func RegisterEnum(typeName string, unusedNameMap map[int32]string, valueMap map[string]int32) {
|
||||
if _, ok := enumValueMaps[typeName]; ok {
|
||||
panic("proto: duplicate enum registered: " + typeName)
|
||||
}
|
||||
enumValueMaps[typeName] = valueMap
|
||||
}
|
||||
|
||||
// EnumValueMap returns the mapping from names to integers of the
|
||||
// enum type enumType, or a nil if not found.
|
||||
func EnumValueMap(enumType string) map[string]int32 {
|
||||
return enumValueMaps[enumType]
|
||||
}
|
||||
|
||||
// A registry of all linked message types.
|
||||
// The string is a fully-qualified proto name ("pkg.Message").
|
||||
var (
|
||||
protoTypes = make(map[string]reflect.Type)
|
||||
revProtoTypes = make(map[reflect.Type]string)
|
||||
)
|
||||
|
||||
// RegisterType is called from generated code and maps from the fully qualified
|
||||
// proto name to the type (pointer to struct) of the protocol buffer.
|
||||
func RegisterType(x Message, name string) {
|
||||
if _, ok := protoTypes[name]; ok {
|
||||
// TODO: Some day, make this a panic.
|
||||
log.Printf("proto: duplicate proto type registered: %s", name)
|
||||
return
|
||||
}
|
||||
t := reflect.TypeOf(x)
|
||||
protoTypes[name] = t
|
||||
revProtoTypes[t] = name
|
||||
}
|
||||
|
||||
// MessageName returns the fully-qualified proto name for the given message type.
|
||||
func MessageName(x Message) string {
|
||||
type xname interface {
|
||||
XXX_MessageName() string
|
||||
}
|
||||
if m, ok := x.(xname); ok {
|
||||
return m.XXX_MessageName()
|
||||
}
|
||||
return revProtoTypes[reflect.TypeOf(x)]
|
||||
}
|
||||
|
||||
// MessageType returns the message type (pointer to struct) for a named message.
|
||||
func MessageType(name string) reflect.Type { return protoTypes[name] }
|
||||
|
||||
// A registry of all linked proto files.
|
||||
var (
|
||||
protoFiles = make(map[string][]byte) // file name => fileDescriptor
|
||||
)
|
||||
|
||||
// RegisterFile is called from generated code and maps from the
|
||||
// full file name of a .proto file to its compressed FileDescriptorProto.
|
||||
func RegisterFile(filename string, fileDescriptor []byte) {
|
||||
protoFiles[filename] = fileDescriptor
|
||||
}
|
||||
|
||||
// FileDescriptor returns the compressed FileDescriptorProto for a .proto file.
|
||||
func FileDescriptor(filename string) []byte { return protoFiles[filename] }
|
||||
+854
@@ -0,0 +1,854 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
package proto
|
||||
|
||||
// Functions for writing the text protocol buffer format.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"math"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var (
|
||||
newline = []byte("\n")
|
||||
spaces = []byte(" ")
|
||||
gtNewline = []byte(">\n")
|
||||
endBraceNewline = []byte("}\n")
|
||||
backslashN = []byte{'\\', 'n'}
|
||||
backslashR = []byte{'\\', 'r'}
|
||||
backslashT = []byte{'\\', 't'}
|
||||
backslashDQ = []byte{'\\', '"'}
|
||||
backslashBS = []byte{'\\', '\\'}
|
||||
posInf = []byte("inf")
|
||||
negInf = []byte("-inf")
|
||||
nan = []byte("nan")
|
||||
)
|
||||
|
||||
type writer interface {
|
||||
io.Writer
|
||||
WriteByte(byte) error
|
||||
}
|
||||
|
||||
// textWriter is an io.Writer that tracks its indentation level.
|
||||
type textWriter struct {
|
||||
ind int
|
||||
complete bool // if the current position is a complete line
|
||||
compact bool // whether to write out as a one-liner
|
||||
w writer
|
||||
}
|
||||
|
||||
func (w *textWriter) WriteString(s string) (n int, err error) {
|
||||
if !strings.Contains(s, "\n") {
|
||||
if !w.compact && w.complete {
|
||||
w.writeIndent()
|
||||
}
|
||||
w.complete = false
|
||||
return io.WriteString(w.w, s)
|
||||
}
|
||||
// WriteString is typically called without newlines, so this
|
||||
// codepath and its copy are rare. We copy to avoid
|
||||
// duplicating all of Write's logic here.
|
||||
return w.Write([]byte(s))
|
||||
}
|
||||
|
||||
func (w *textWriter) Write(p []byte) (n int, err error) {
|
||||
newlines := bytes.Count(p, newline)
|
||||
if newlines == 0 {
|
||||
if !w.compact && w.complete {
|
||||
w.writeIndent()
|
||||
}
|
||||
n, err = w.w.Write(p)
|
||||
w.complete = false
|
||||
return n, err
|
||||
}
|
||||
|
||||
frags := bytes.SplitN(p, newline, newlines+1)
|
||||
if w.compact {
|
||||
for i, frag := range frags {
|
||||
if i > 0 {
|
||||
if err := w.w.WriteByte(' '); err != nil {
|
||||
return n, err
|
||||
}
|
||||
n++
|
||||
}
|
||||
nn, err := w.w.Write(frag)
|
||||
n += nn
|
||||
if err != nil {
|
||||
return n, err
|
||||
}
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
for i, frag := range frags {
|
||||
if w.complete {
|
||||
w.writeIndent()
|
||||
}
|
||||
nn, err := w.w.Write(frag)
|
||||
n += nn
|
||||
if err != nil {
|
||||
return n, err
|
||||
}
|
||||
if i+1 < len(frags) {
|
||||
if err := w.w.WriteByte('\n'); err != nil {
|
||||
return n, err
|
||||
}
|
||||
n++
|
||||
}
|
||||
}
|
||||
w.complete = len(frags[len(frags)-1]) == 0
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func (w *textWriter) WriteByte(c byte) error {
|
||||
if w.compact && c == '\n' {
|
||||
c = ' '
|
||||
}
|
||||
if !w.compact && w.complete {
|
||||
w.writeIndent()
|
||||
}
|
||||
err := w.w.WriteByte(c)
|
||||
w.complete = c == '\n'
|
||||
return err
|
||||
}
|
||||
|
||||
func (w *textWriter) indent() { w.ind++ }
|
||||
|
||||
func (w *textWriter) unindent() {
|
||||
if w.ind == 0 {
|
||||
log.Print("proto: textWriter unindented too far")
|
||||
return
|
||||
}
|
||||
w.ind--
|
||||
}
|
||||
|
||||
func writeName(w *textWriter, props *Properties) error {
|
||||
if _, err := w.WriteString(props.OrigName); err != nil {
|
||||
return err
|
||||
}
|
||||
if props.Wire != "group" {
|
||||
return w.WriteByte(':')
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// raw is the interface satisfied by RawMessage.
|
||||
type raw interface {
|
||||
Bytes() []byte
|
||||
}
|
||||
|
||||
func requiresQuotes(u string) bool {
|
||||
// When type URL contains any characters except [0-9A-Za-z./\-]*, it must be quoted.
|
||||
for _, ch := range u {
|
||||
switch {
|
||||
case ch == '.' || ch == '/' || ch == '_':
|
||||
continue
|
||||
case '0' <= ch && ch <= '9':
|
||||
continue
|
||||
case 'A' <= ch && ch <= 'Z':
|
||||
continue
|
||||
case 'a' <= ch && ch <= 'z':
|
||||
continue
|
||||
default:
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// isAny reports whether sv is a google.protobuf.Any message
|
||||
func isAny(sv reflect.Value) bool {
|
||||
type wkt interface {
|
||||
XXX_WellKnownType() string
|
||||
}
|
||||
t, ok := sv.Addr().Interface().(wkt)
|
||||
return ok && t.XXX_WellKnownType() == "Any"
|
||||
}
|
||||
|
||||
// writeProto3Any writes an expanded google.protobuf.Any message.
|
||||
//
|
||||
// It returns (false, nil) if sv value can't be unmarshaled (e.g. because
|
||||
// required messages are not linked in).
|
||||
//
|
||||
// It returns (true, error) when sv was written in expanded format or an error
|
||||
// was encountered.
|
||||
func (tm *TextMarshaler) writeProto3Any(w *textWriter, sv reflect.Value) (bool, error) {
|
||||
turl := sv.FieldByName("TypeUrl")
|
||||
val := sv.FieldByName("Value")
|
||||
if !turl.IsValid() || !val.IsValid() {
|
||||
return true, errors.New("proto: invalid google.protobuf.Any message")
|
||||
}
|
||||
|
||||
b, ok := val.Interface().([]byte)
|
||||
if !ok {
|
||||
return true, errors.New("proto: invalid google.protobuf.Any message")
|
||||
}
|
||||
|
||||
parts := strings.Split(turl.String(), "/")
|
||||
mt := MessageType(parts[len(parts)-1])
|
||||
if mt == nil {
|
||||
return false, nil
|
||||
}
|
||||
m := reflect.New(mt.Elem())
|
||||
if err := Unmarshal(b, m.Interface().(Message)); err != nil {
|
||||
return false, nil
|
||||
}
|
||||
w.Write([]byte("["))
|
||||
u := turl.String()
|
||||
if requiresQuotes(u) {
|
||||
writeString(w, u)
|
||||
} else {
|
||||
w.Write([]byte(u))
|
||||
}
|
||||
if w.compact {
|
||||
w.Write([]byte("]:<"))
|
||||
} else {
|
||||
w.Write([]byte("]: <\n"))
|
||||
w.ind++
|
||||
}
|
||||
if err := tm.writeStruct(w, m.Elem()); err != nil {
|
||||
return true, err
|
||||
}
|
||||
if w.compact {
|
||||
w.Write([]byte("> "))
|
||||
} else {
|
||||
w.ind--
|
||||
w.Write([]byte(">\n"))
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func (tm *TextMarshaler) writeStruct(w *textWriter, sv reflect.Value) error {
|
||||
if tm.ExpandAny && isAny(sv) {
|
||||
if canExpand, err := tm.writeProto3Any(w, sv); canExpand {
|
||||
return err
|
||||
}
|
||||
}
|
||||
st := sv.Type()
|
||||
sprops := GetProperties(st)
|
||||
for i := 0; i < sv.NumField(); i++ {
|
||||
fv := sv.Field(i)
|
||||
props := sprops.Prop[i]
|
||||
name := st.Field(i).Name
|
||||
|
||||
if strings.HasPrefix(name, "XXX_") {
|
||||
// There are two XXX_ fields:
|
||||
// XXX_unrecognized []byte
|
||||
// XXX_extensions map[int32]proto.Extension
|
||||
// The first is handled here;
|
||||
// the second is handled at the bottom of this function.
|
||||
if name == "XXX_unrecognized" && !fv.IsNil() {
|
||||
if err := writeUnknownStruct(w, fv.Interface().([]byte)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if fv.Kind() == reflect.Ptr && fv.IsNil() {
|
||||
// Field not filled in. This could be an optional field or
|
||||
// a required field that wasn't filled in. Either way, there
|
||||
// isn't anything we can show for it.
|
||||
continue
|
||||
}
|
||||
if fv.Kind() == reflect.Slice && fv.IsNil() {
|
||||
// Repeated field that is empty, or a bytes field that is unused.
|
||||
continue
|
||||
}
|
||||
|
||||
if props.Repeated && fv.Kind() == reflect.Slice {
|
||||
// Repeated field.
|
||||
for j := 0; j < fv.Len(); j++ {
|
||||
if err := writeName(w, props); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
v := fv.Index(j)
|
||||
if v.Kind() == reflect.Ptr && v.IsNil() {
|
||||
// A nil message in a repeated field is not valid,
|
||||
// but we can handle that more gracefully than panicking.
|
||||
if _, err := w.Write([]byte("<nil>\n")); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err := tm.writeAny(w, v, props); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if fv.Kind() == reflect.Map {
|
||||
// Map fields are rendered as a repeated struct with key/value fields.
|
||||
keys := fv.MapKeys()
|
||||
sort.Sort(mapKeys(keys))
|
||||
for _, key := range keys {
|
||||
val := fv.MapIndex(key)
|
||||
if err := writeName(w, props); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// open struct
|
||||
if err := w.WriteByte('<'); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
w.indent()
|
||||
// key
|
||||
if _, err := w.WriteString("key:"); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := tm.writeAny(w, key, props.mkeyprop); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
// nil values aren't legal, but we can avoid panicking because of them.
|
||||
if val.Kind() != reflect.Ptr || !val.IsNil() {
|
||||
// value
|
||||
if _, err := w.WriteString("value:"); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := tm.writeAny(w, val, props.mvalprop); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// close struct
|
||||
w.unindent()
|
||||
if err := w.WriteByte('>'); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if props.proto3 && fv.Kind() == reflect.Slice && fv.Len() == 0 {
|
||||
// empty bytes field
|
||||
continue
|
||||
}
|
||||
if fv.Kind() != reflect.Ptr && fv.Kind() != reflect.Slice {
|
||||
// proto3 non-repeated scalar field; skip if zero value
|
||||
if isProto3Zero(fv) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
if fv.Kind() == reflect.Interface {
|
||||
// Check if it is a oneof.
|
||||
if st.Field(i).Tag.Get("protobuf_oneof") != "" {
|
||||
// fv is nil, or holds a pointer to generated struct.
|
||||
// That generated struct has exactly one field,
|
||||
// which has a protobuf struct tag.
|
||||
if fv.IsNil() {
|
||||
continue
|
||||
}
|
||||
inner := fv.Elem().Elem() // interface -> *T -> T
|
||||
tag := inner.Type().Field(0).Tag.Get("protobuf")
|
||||
props = new(Properties) // Overwrite the outer props var, but not its pointee.
|
||||
props.Parse(tag)
|
||||
// Write the value in the oneof, not the oneof itself.
|
||||
fv = inner.Field(0)
|
||||
|
||||
// Special case to cope with malformed messages gracefully:
|
||||
// If the value in the oneof is a nil pointer, don't panic
|
||||
// in writeAny.
|
||||
if fv.Kind() == reflect.Ptr && fv.IsNil() {
|
||||
// Use errors.New so writeAny won't render quotes.
|
||||
msg := errors.New("/* nil */")
|
||||
fv = reflect.ValueOf(&msg).Elem()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if err := writeName(w, props); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if b, ok := fv.Interface().(raw); ok {
|
||||
if err := writeRaw(w, b.Bytes()); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Enums have a String method, so writeAny will work fine.
|
||||
if err := tm.writeAny(w, fv, props); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// Extensions (the XXX_extensions field).
|
||||
pv := sv.Addr()
|
||||
if _, ok := extendable(pv.Interface()); ok {
|
||||
if err := tm.writeExtensions(w, pv); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// writeRaw writes an uninterpreted raw message.
|
||||
func writeRaw(w *textWriter, b []byte) error {
|
||||
if err := w.WriteByte('<'); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
w.indent()
|
||||
if err := writeUnknownStruct(w, b); err != nil {
|
||||
return err
|
||||
}
|
||||
w.unindent()
|
||||
if err := w.WriteByte('>'); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// writeAny writes an arbitrary field.
|
||||
func (tm *TextMarshaler) writeAny(w *textWriter, v reflect.Value, props *Properties) error {
|
||||
v = reflect.Indirect(v)
|
||||
|
||||
// Floats have special cases.
|
||||
if v.Kind() == reflect.Float32 || v.Kind() == reflect.Float64 {
|
||||
x := v.Float()
|
||||
var b []byte
|
||||
switch {
|
||||
case math.IsInf(x, 1):
|
||||
b = posInf
|
||||
case math.IsInf(x, -1):
|
||||
b = negInf
|
||||
case math.IsNaN(x):
|
||||
b = nan
|
||||
}
|
||||
if b != nil {
|
||||
_, err := w.Write(b)
|
||||
return err
|
||||
}
|
||||
// Other values are handled below.
|
||||
}
|
||||
|
||||
// We don't attempt to serialise every possible value type; only those
|
||||
// that can occur in protocol buffers.
|
||||
switch v.Kind() {
|
||||
case reflect.Slice:
|
||||
// Should only be a []byte; repeated fields are handled in writeStruct.
|
||||
if err := writeString(w, string(v.Bytes())); err != nil {
|
||||
return err
|
||||
}
|
||||
case reflect.String:
|
||||
if err := writeString(w, v.String()); err != nil {
|
||||
return err
|
||||
}
|
||||
case reflect.Struct:
|
||||
// Required/optional group/message.
|
||||
var bra, ket byte = '<', '>'
|
||||
if props != nil && props.Wire == "group" {
|
||||
bra, ket = '{', '}'
|
||||
}
|
||||
if err := w.WriteByte(bra); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
w.indent()
|
||||
if etm, ok := v.Interface().(encoding.TextMarshaler); ok {
|
||||
text, err := etm.MarshalText()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err = w.Write(text); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if err := tm.writeStruct(w, v); err != nil {
|
||||
return err
|
||||
}
|
||||
w.unindent()
|
||||
if err := w.WriteByte(ket); err != nil {
|
||||
return err
|
||||
}
|
||||
default:
|
||||
_, err := fmt.Fprint(w, v.Interface())
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// equivalent to C's isprint.
|
||||
func isprint(c byte) bool {
|
||||
return c >= 0x20 && c < 0x7f
|
||||
}
|
||||
|
||||
// writeString writes a string in the protocol buffer text format.
|
||||
// It is similar to strconv.Quote except we don't use Go escape sequences,
|
||||
// we treat the string as a byte sequence, and we use octal escapes.
|
||||
// These differences are to maintain interoperability with the other
|
||||
// languages' implementations of the text format.
|
||||
func writeString(w *textWriter, s string) error {
|
||||
// use WriteByte here to get any needed indent
|
||||
if err := w.WriteByte('"'); err != nil {
|
||||
return err
|
||||
}
|
||||
// Loop over the bytes, not the runes.
|
||||
for i := 0; i < len(s); i++ {
|
||||
var err error
|
||||
// Divergence from C++: we don't escape apostrophes.
|
||||
// There's no need to escape them, and the C++ parser
|
||||
// copes with a naked apostrophe.
|
||||
switch c := s[i]; c {
|
||||
case '\n':
|
||||
_, err = w.w.Write(backslashN)
|
||||
case '\r':
|
||||
_, err = w.w.Write(backslashR)
|
||||
case '\t':
|
||||
_, err = w.w.Write(backslashT)
|
||||
case '"':
|
||||
_, err = w.w.Write(backslashDQ)
|
||||
case '\\':
|
||||
_, err = w.w.Write(backslashBS)
|
||||
default:
|
||||
if isprint(c) {
|
||||
err = w.w.WriteByte(c)
|
||||
} else {
|
||||
_, err = fmt.Fprintf(w.w, "\\%03o", c)
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return w.WriteByte('"')
|
||||
}
|
||||
|
||||
func writeUnknownStruct(w *textWriter, data []byte) (err error) {
|
||||
if !w.compact {
|
||||
if _, err := fmt.Fprintf(w, "/* %d unknown bytes */\n", len(data)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
b := NewBuffer(data)
|
||||
for b.index < len(b.buf) {
|
||||
x, err := b.DecodeVarint()
|
||||
if err != nil {
|
||||
_, err := fmt.Fprintf(w, "/* %v */\n", err)
|
||||
return err
|
||||
}
|
||||
wire, tag := x&7, x>>3
|
||||
if wire == WireEndGroup {
|
||||
w.unindent()
|
||||
if _, err := w.Write(endBraceNewline); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if _, err := fmt.Fprint(w, tag); err != nil {
|
||||
return err
|
||||
}
|
||||
if wire != WireStartGroup {
|
||||
if err := w.WriteByte(':'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if !w.compact || wire == WireStartGroup {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
switch wire {
|
||||
case WireBytes:
|
||||
buf, e := b.DecodeRawBytes(false)
|
||||
if e == nil {
|
||||
_, err = fmt.Fprintf(w, "%q", buf)
|
||||
} else {
|
||||
_, err = fmt.Fprintf(w, "/* %v */", e)
|
||||
}
|
||||
case WireFixed32:
|
||||
x, err = b.DecodeFixed32()
|
||||
err = writeUnknownInt(w, x, err)
|
||||
case WireFixed64:
|
||||
x, err = b.DecodeFixed64()
|
||||
err = writeUnknownInt(w, x, err)
|
||||
case WireStartGroup:
|
||||
err = w.WriteByte('{')
|
||||
w.indent()
|
||||
case WireVarint:
|
||||
x, err = b.DecodeVarint()
|
||||
err = writeUnknownInt(w, x, err)
|
||||
default:
|
||||
_, err = fmt.Fprintf(w, "/* unknown wire type %d */", wire)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err = w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeUnknownInt(w *textWriter, x uint64, err error) error {
|
||||
if err == nil {
|
||||
_, err = fmt.Fprint(w, x)
|
||||
} else {
|
||||
_, err = fmt.Fprintf(w, "/* %v */", err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
type int32Slice []int32
|
||||
|
||||
func (s int32Slice) Len() int { return len(s) }
|
||||
func (s int32Slice) Less(i, j int) bool { return s[i] < s[j] }
|
||||
func (s int32Slice) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
|
||||
|
||||
// writeExtensions writes all the extensions in pv.
|
||||
// pv is assumed to be a pointer to a protocol message struct that is extendable.
|
||||
func (tm *TextMarshaler) writeExtensions(w *textWriter, pv reflect.Value) error {
|
||||
emap := extensionMaps[pv.Type().Elem()]
|
||||
ep, _ := extendable(pv.Interface())
|
||||
|
||||
// Order the extensions by ID.
|
||||
// This isn't strictly necessary, but it will give us
|
||||
// canonical output, which will also make testing easier.
|
||||
m, mu := ep.extensionsRead()
|
||||
if m == nil {
|
||||
return nil
|
||||
}
|
||||
mu.Lock()
|
||||
ids := make([]int32, 0, len(m))
|
||||
for id := range m {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Sort(int32Slice(ids))
|
||||
mu.Unlock()
|
||||
|
||||
for _, extNum := range ids {
|
||||
ext := m[extNum]
|
||||
var desc *ExtensionDesc
|
||||
if emap != nil {
|
||||
desc = emap[extNum]
|
||||
}
|
||||
if desc == nil {
|
||||
// Unknown extension.
|
||||
if err := writeUnknownStruct(w, ext.enc); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
pb, err := GetExtension(ep, desc)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed getting extension: %v", err)
|
||||
}
|
||||
|
||||
// Repeated extensions will appear as a slice.
|
||||
if !desc.repeated() {
|
||||
if err := tm.writeExtension(w, desc.Name, pb); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
v := reflect.ValueOf(pb)
|
||||
for i := 0; i < v.Len(); i++ {
|
||||
if err := tm.writeExtension(w, desc.Name, v.Index(i).Interface()); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (tm *TextMarshaler) writeExtension(w *textWriter, name string, pb interface{}) error {
|
||||
if _, err := fmt.Fprintf(w, "[%s]:", name); err != nil {
|
||||
return err
|
||||
}
|
||||
if !w.compact {
|
||||
if err := w.WriteByte(' '); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := tm.writeAny(w, reflect.ValueOf(pb), nil); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := w.WriteByte('\n'); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (w *textWriter) writeIndent() {
|
||||
if !w.complete {
|
||||
return
|
||||
}
|
||||
remain := w.ind * 2
|
||||
for remain > 0 {
|
||||
n := remain
|
||||
if n > len(spaces) {
|
||||
n = len(spaces)
|
||||
}
|
||||
w.w.Write(spaces[:n])
|
||||
remain -= n
|
||||
}
|
||||
w.complete = false
|
||||
}
|
||||
|
||||
// TextMarshaler is a configurable text format marshaler.
|
||||
type TextMarshaler struct {
|
||||
Compact bool // use compact text format (one line).
|
||||
ExpandAny bool // expand google.protobuf.Any messages of known types
|
||||
}
|
||||
|
||||
// Marshal writes a given protocol buffer in text format.
|
||||
// The only errors returned are from w.
|
||||
func (tm *TextMarshaler) Marshal(w io.Writer, pb Message) error {
|
||||
val := reflect.ValueOf(pb)
|
||||
if pb == nil || val.IsNil() {
|
||||
w.Write([]byte("<nil>"))
|
||||
return nil
|
||||
}
|
||||
var bw *bufio.Writer
|
||||
ww, ok := w.(writer)
|
||||
if !ok {
|
||||
bw = bufio.NewWriter(w)
|
||||
ww = bw
|
||||
}
|
||||
aw := &textWriter{
|
||||
w: ww,
|
||||
complete: true,
|
||||
compact: tm.Compact,
|
||||
}
|
||||
|
||||
if etm, ok := pb.(encoding.TextMarshaler); ok {
|
||||
text, err := etm.MarshalText()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err = aw.Write(text); err != nil {
|
||||
return err
|
||||
}
|
||||
if bw != nil {
|
||||
return bw.Flush()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
// Dereference the received pointer so we don't have outer < and >.
|
||||
v := reflect.Indirect(val)
|
||||
if err := tm.writeStruct(aw, v); err != nil {
|
||||
return err
|
||||
}
|
||||
if bw != nil {
|
||||
return bw.Flush()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Text is the same as Marshal, but returns the string directly.
|
||||
func (tm *TextMarshaler) Text(pb Message) string {
|
||||
var buf bytes.Buffer
|
||||
tm.Marshal(&buf, pb)
|
||||
return buf.String()
|
||||
}
|
||||
|
||||
var (
|
||||
defaultTextMarshaler = TextMarshaler{}
|
||||
compactTextMarshaler = TextMarshaler{Compact: true}
|
||||
)
|
||||
|
||||
// TODO: consider removing some of the Marshal functions below.
|
||||
|
||||
// MarshalText writes a given protocol buffer in text format.
|
||||
// The only errors returned are from w.
|
||||
func MarshalText(w io.Writer, pb Message) error { return defaultTextMarshaler.Marshal(w, pb) }
|
||||
|
||||
// MarshalTextString is the same as MarshalText, but returns the string directly.
|
||||
func MarshalTextString(pb Message) string { return defaultTextMarshaler.Text(pb) }
|
||||
|
||||
// CompactText writes a given protocol buffer in compact text format (one line).
|
||||
func CompactText(w io.Writer, pb Message) error { return compactTextMarshaler.Marshal(w, pb) }
|
||||
|
||||
// CompactTextString is the same as CompactText, but returns the string directly.
|
||||
func CompactTextString(pb Message) string { return compactTextMarshaler.Text(pb) }
|
||||
+895
@@ -0,0 +1,895 @@
|
||||
// Go support for Protocol Buffers - Google's data interchange format
|
||||
//
|
||||
// Copyright 2010 The Go Authors. All rights reserved.
|
||||
// https://github.com/golang/protobuf
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
package proto
|
||||
|
||||
// Functions for parsing the Text protocol buffer format.
|
||||
// TODO: message sets.
|
||||
|
||||
import (
|
||||
"encoding"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// Error string emitted when deserializing Any and fields are already set
|
||||
const anyRepeatedlyUnpacked = "Any message unpacked multiple times, or %q already set"
|
||||
|
||||
type ParseError struct {
|
||||
Message string
|
||||
Line int // 1-based line number
|
||||
Offset int // 0-based byte offset from start of input
|
||||
}
|
||||
|
||||
func (p *ParseError) Error() string {
|
||||
if p.Line == 1 {
|
||||
// show offset only for first line
|
||||
return fmt.Sprintf("line 1.%d: %v", p.Offset, p.Message)
|
||||
}
|
||||
return fmt.Sprintf("line %d: %v", p.Line, p.Message)
|
||||
}
|
||||
|
||||
type token struct {
|
||||
value string
|
||||
err *ParseError
|
||||
line int // line number
|
||||
offset int // byte number from start of input, not start of line
|
||||
unquoted string // the unquoted version of value, if it was a quoted string
|
||||
}
|
||||
|
||||
func (t *token) String() string {
|
||||
if t.err == nil {
|
||||
return fmt.Sprintf("%q (line=%d, offset=%d)", t.value, t.line, t.offset)
|
||||
}
|
||||
return fmt.Sprintf("parse error: %v", t.err)
|
||||
}
|
||||
|
||||
type textParser struct {
|
||||
s string // remaining input
|
||||
done bool // whether the parsing is finished (success or error)
|
||||
backed bool // whether back() was called
|
||||
offset, line int
|
||||
cur token
|
||||
}
|
||||
|
||||
func newTextParser(s string) *textParser {
|
||||
p := new(textParser)
|
||||
p.s = s
|
||||
p.line = 1
|
||||
p.cur.line = 1
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *textParser) errorf(format string, a ...interface{}) *ParseError {
|
||||
pe := &ParseError{fmt.Sprintf(format, a...), p.cur.line, p.cur.offset}
|
||||
p.cur.err = pe
|
||||
p.done = true
|
||||
return pe
|
||||
}
|
||||
|
||||
// Numbers and identifiers are matched by [-+._A-Za-z0-9]
|
||||
func isIdentOrNumberChar(c byte) bool {
|
||||
switch {
|
||||
case 'A' <= c && c <= 'Z', 'a' <= c && c <= 'z':
|
||||
return true
|
||||
case '0' <= c && c <= '9':
|
||||
return true
|
||||
}
|
||||
switch c {
|
||||
case '-', '+', '.', '_':
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isWhitespace(c byte) bool {
|
||||
switch c {
|
||||
case ' ', '\t', '\n', '\r':
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isQuote(c byte) bool {
|
||||
switch c {
|
||||
case '"', '\'':
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (p *textParser) skipWhitespace() {
|
||||
i := 0
|
||||
for i < len(p.s) && (isWhitespace(p.s[i]) || p.s[i] == '#') {
|
||||
if p.s[i] == '#' {
|
||||
// comment; skip to end of line or input
|
||||
for i < len(p.s) && p.s[i] != '\n' {
|
||||
i++
|
||||
}
|
||||
if i == len(p.s) {
|
||||
break
|
||||
}
|
||||
}
|
||||
if p.s[i] == '\n' {
|
||||
p.line++
|
||||
}
|
||||
i++
|
||||
}
|
||||
p.offset += i
|
||||
p.s = p.s[i:len(p.s)]
|
||||
if len(p.s) == 0 {
|
||||
p.done = true
|
||||
}
|
||||
}
|
||||
|
||||
func (p *textParser) advance() {
|
||||
// Skip whitespace
|
||||
p.skipWhitespace()
|
||||
if p.done {
|
||||
return
|
||||
}
|
||||
|
||||
// Start of non-whitespace
|
||||
p.cur.err = nil
|
||||
p.cur.offset, p.cur.line = p.offset, p.line
|
||||
p.cur.unquoted = ""
|
||||
switch p.s[0] {
|
||||
case '<', '>', '{', '}', ':', '[', ']', ';', ',', '/':
|
||||
// Single symbol
|
||||
p.cur.value, p.s = p.s[0:1], p.s[1:len(p.s)]
|
||||
case '"', '\'':
|
||||
// Quoted string
|
||||
i := 1
|
||||
for i < len(p.s) && p.s[i] != p.s[0] && p.s[i] != '\n' {
|
||||
if p.s[i] == '\\' && i+1 < len(p.s) {
|
||||
// skip escaped char
|
||||
i++
|
||||
}
|
||||
i++
|
||||
}
|
||||
if i >= len(p.s) || p.s[i] != p.s[0] {
|
||||
p.errorf("unmatched quote")
|
||||
return
|
||||
}
|
||||
unq, err := unquoteC(p.s[1:i], rune(p.s[0]))
|
||||
if err != nil {
|
||||
p.errorf("invalid quoted string %s: %v", p.s[0:i+1], err)
|
||||
return
|
||||
}
|
||||
p.cur.value, p.s = p.s[0:i+1], p.s[i+1:len(p.s)]
|
||||
p.cur.unquoted = unq
|
||||
default:
|
||||
i := 0
|
||||
for i < len(p.s) && isIdentOrNumberChar(p.s[i]) {
|
||||
i++
|
||||
}
|
||||
if i == 0 {
|
||||
p.errorf("unexpected byte %#x", p.s[0])
|
||||
return
|
||||
}
|
||||
p.cur.value, p.s = p.s[0:i], p.s[i:len(p.s)]
|
||||
}
|
||||
p.offset += len(p.cur.value)
|
||||
}
|
||||
|
||||
var (
|
||||
errBadUTF8 = errors.New("proto: bad UTF-8")
|
||||
errBadHex = errors.New("proto: bad hexadecimal")
|
||||
)
|
||||
|
||||
func unquoteC(s string, quote rune) (string, error) {
|
||||
// This is based on C++'s tokenizer.cc.
|
||||
// Despite its name, this is *not* parsing C syntax.
|
||||
// For instance, "\0" is an invalid quoted string.
|
||||
|
||||
// Avoid allocation in trivial cases.
|
||||
simple := true
|
||||
for _, r := range s {
|
||||
if r == '\\' || r == quote {
|
||||
simple = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if simple {
|
||||
return s, nil
|
||||
}
|
||||
|
||||
buf := make([]byte, 0, 3*len(s)/2)
|
||||
for len(s) > 0 {
|
||||
r, n := utf8.DecodeRuneInString(s)
|
||||
if r == utf8.RuneError && n == 1 {
|
||||
return "", errBadUTF8
|
||||
}
|
||||
s = s[n:]
|
||||
if r != '\\' {
|
||||
if r < utf8.RuneSelf {
|
||||
buf = append(buf, byte(r))
|
||||
} else {
|
||||
buf = append(buf, string(r)...)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
ch, tail, err := unescape(s)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
buf = append(buf, ch...)
|
||||
s = tail
|
||||
}
|
||||
return string(buf), nil
|
||||
}
|
||||
|
||||
func unescape(s string) (ch string, tail string, err error) {
|
||||
r, n := utf8.DecodeRuneInString(s)
|
||||
if r == utf8.RuneError && n == 1 {
|
||||
return "", "", errBadUTF8
|
||||
}
|
||||
s = s[n:]
|
||||
switch r {
|
||||
case 'a':
|
||||
return "\a", s, nil
|
||||
case 'b':
|
||||
return "\b", s, nil
|
||||
case 'f':
|
||||
return "\f", s, nil
|
||||
case 'n':
|
||||
return "\n", s, nil
|
||||
case 'r':
|
||||
return "\r", s, nil
|
||||
case 't':
|
||||
return "\t", s, nil
|
||||
case 'v':
|
||||
return "\v", s, nil
|
||||
case '?':
|
||||
return "?", s, nil // trigraph workaround
|
||||
case '\'', '"', '\\':
|
||||
return string(r), s, nil
|
||||
case '0', '1', '2', '3', '4', '5', '6', '7', 'x', 'X':
|
||||
if len(s) < 2 {
|
||||
return "", "", fmt.Errorf(`\%c requires 2 following digits`, r)
|
||||
}
|
||||
base := 8
|
||||
ss := s[:2]
|
||||
s = s[2:]
|
||||
if r == 'x' || r == 'X' {
|
||||
base = 16
|
||||
} else {
|
||||
ss = string(r) + ss
|
||||
}
|
||||
i, err := strconv.ParseUint(ss, base, 8)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
return string([]byte{byte(i)}), s, nil
|
||||
case 'u', 'U':
|
||||
n := 4
|
||||
if r == 'U' {
|
||||
n = 8
|
||||
}
|
||||
if len(s) < n {
|
||||
return "", "", fmt.Errorf(`\%c requires %d digits`, r, n)
|
||||
}
|
||||
|
||||
bs := make([]byte, n/2)
|
||||
for i := 0; i < n; i += 2 {
|
||||
a, ok1 := unhex(s[i])
|
||||
b, ok2 := unhex(s[i+1])
|
||||
if !ok1 || !ok2 {
|
||||
return "", "", errBadHex
|
||||
}
|
||||
bs[i/2] = a<<4 | b
|
||||
}
|
||||
s = s[n:]
|
||||
return string(bs), s, nil
|
||||
}
|
||||
return "", "", fmt.Errorf(`unknown escape \%c`, r)
|
||||
}
|
||||
|
||||
// Adapted from src/pkg/strconv/quote.go.
|
||||
func unhex(b byte) (v byte, ok bool) {
|
||||
switch {
|
||||
case '0' <= b && b <= '9':
|
||||
return b - '0', true
|
||||
case 'a' <= b && b <= 'f':
|
||||
return b - 'a' + 10, true
|
||||
case 'A' <= b && b <= 'F':
|
||||
return b - 'A' + 10, true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
// Back off the parser by one token. Can only be done between calls to next().
|
||||
// It makes the next advance() a no-op.
|
||||
func (p *textParser) back() { p.backed = true }
|
||||
|
||||
// Advances the parser and returns the new current token.
|
||||
func (p *textParser) next() *token {
|
||||
if p.backed || p.done {
|
||||
p.backed = false
|
||||
return &p.cur
|
||||
}
|
||||
p.advance()
|
||||
if p.done {
|
||||
p.cur.value = ""
|
||||
} else if len(p.cur.value) > 0 && isQuote(p.cur.value[0]) {
|
||||
// Look for multiple quoted strings separated by whitespace,
|
||||
// and concatenate them.
|
||||
cat := p.cur
|
||||
for {
|
||||
p.skipWhitespace()
|
||||
if p.done || !isQuote(p.s[0]) {
|
||||
break
|
||||
}
|
||||
p.advance()
|
||||
if p.cur.err != nil {
|
||||
return &p.cur
|
||||
}
|
||||
cat.value += " " + p.cur.value
|
||||
cat.unquoted += p.cur.unquoted
|
||||
}
|
||||
p.done = false // parser may have seen EOF, but we want to return cat
|
||||
p.cur = cat
|
||||
}
|
||||
return &p.cur
|
||||
}
|
||||
|
||||
func (p *textParser) consumeToken(s string) error {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value != s {
|
||||
p.back()
|
||||
return p.errorf("expected %q, found %q", s, tok.value)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Return a RequiredNotSetError indicating which required field was not set.
|
||||
func (p *textParser) missingRequiredFieldError(sv reflect.Value) *RequiredNotSetError {
|
||||
st := sv.Type()
|
||||
sprops := GetProperties(st)
|
||||
for i := 0; i < st.NumField(); i++ {
|
||||
if !isNil(sv.Field(i)) {
|
||||
continue
|
||||
}
|
||||
|
||||
props := sprops.Prop[i]
|
||||
if props.Required {
|
||||
return &RequiredNotSetError{fmt.Sprintf("%v.%v", st, props.OrigName)}
|
||||
}
|
||||
}
|
||||
return &RequiredNotSetError{fmt.Sprintf("%v.<unknown field name>", st)} // should not happen
|
||||
}
|
||||
|
||||
// Returns the index in the struct for the named field, as well as the parsed tag properties.
|
||||
func structFieldByName(sprops *StructProperties, name string) (int, *Properties, bool) {
|
||||
i, ok := sprops.decoderOrigNames[name]
|
||||
if ok {
|
||||
return i, sprops.Prop[i], true
|
||||
}
|
||||
return -1, nil, false
|
||||
}
|
||||
|
||||
// Consume a ':' from the input stream (if the next token is a colon),
|
||||
// returning an error if a colon is needed but not present.
|
||||
func (p *textParser) checkForColon(props *Properties, typ reflect.Type) *ParseError {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value != ":" {
|
||||
// Colon is optional when the field is a group or message.
|
||||
needColon := true
|
||||
switch props.Wire {
|
||||
case "group":
|
||||
needColon = false
|
||||
case "bytes":
|
||||
// A "bytes" field is either a message, a string, or a repeated field;
|
||||
// those three become *T, *string and []T respectively, so we can check for
|
||||
// this field being a pointer to a non-string.
|
||||
if typ.Kind() == reflect.Ptr {
|
||||
// *T or *string
|
||||
if typ.Elem().Kind() == reflect.String {
|
||||
break
|
||||
}
|
||||
} else if typ.Kind() == reflect.Slice {
|
||||
// []T or []*T
|
||||
if typ.Elem().Kind() != reflect.Ptr {
|
||||
break
|
||||
}
|
||||
} else if typ.Kind() == reflect.String {
|
||||
// The proto3 exception is for a string field,
|
||||
// which requires a colon.
|
||||
break
|
||||
}
|
||||
needColon = false
|
||||
}
|
||||
if needColon {
|
||||
return p.errorf("expected ':', found %q", tok.value)
|
||||
}
|
||||
p.back()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *textParser) readStruct(sv reflect.Value, terminator string) error {
|
||||
st := sv.Type()
|
||||
sprops := GetProperties(st)
|
||||
reqCount := sprops.reqCount
|
||||
var reqFieldErr error
|
||||
fieldSet := make(map[string]bool)
|
||||
// A struct is a sequence of "name: value", terminated by one of
|
||||
// '>' or '}', or the end of the input. A name may also be
|
||||
// "[extension]" or "[type/url]".
|
||||
//
|
||||
// The whole struct can also be an expanded Any message, like:
|
||||
// [type/url] < ... struct contents ... >
|
||||
for {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value == terminator {
|
||||
break
|
||||
}
|
||||
if tok.value == "[" {
|
||||
// Looks like an extension or an Any.
|
||||
//
|
||||
// TODO: Check whether we need to handle
|
||||
// namespace rooted names (e.g. ".something.Foo").
|
||||
extName, err := p.consumeExtName()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if s := strings.LastIndex(extName, "/"); s >= 0 {
|
||||
// If it contains a slash, it's an Any type URL.
|
||||
messageName := extName[s+1:]
|
||||
mt := MessageType(messageName)
|
||||
if mt == nil {
|
||||
return p.errorf("unrecognized message %q in google.protobuf.Any", messageName)
|
||||
}
|
||||
tok = p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
// consume an optional colon
|
||||
if tok.value == ":" {
|
||||
tok = p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
}
|
||||
var terminator string
|
||||
switch tok.value {
|
||||
case "<":
|
||||
terminator = ">"
|
||||
case "{":
|
||||
terminator = "}"
|
||||
default:
|
||||
return p.errorf("expected '{' or '<', found %q", tok.value)
|
||||
}
|
||||
v := reflect.New(mt.Elem())
|
||||
if pe := p.readStruct(v.Elem(), terminator); pe != nil {
|
||||
return pe
|
||||
}
|
||||
b, err := Marshal(v.Interface().(Message))
|
||||
if err != nil {
|
||||
return p.errorf("failed to marshal message of type %q: %v", messageName, err)
|
||||
}
|
||||
if fieldSet["type_url"] {
|
||||
return p.errorf(anyRepeatedlyUnpacked, "type_url")
|
||||
}
|
||||
if fieldSet["value"] {
|
||||
return p.errorf(anyRepeatedlyUnpacked, "value")
|
||||
}
|
||||
sv.FieldByName("TypeUrl").SetString(extName)
|
||||
sv.FieldByName("Value").SetBytes(b)
|
||||
fieldSet["type_url"] = true
|
||||
fieldSet["value"] = true
|
||||
continue
|
||||
}
|
||||
|
||||
var desc *ExtensionDesc
|
||||
// This could be faster, but it's functional.
|
||||
// TODO: Do something smarter than a linear scan.
|
||||
for _, d := range RegisteredExtensions(reflect.New(st).Interface().(Message)) {
|
||||
if d.Name == extName {
|
||||
desc = d
|
||||
break
|
||||
}
|
||||
}
|
||||
if desc == nil {
|
||||
return p.errorf("unrecognized extension %q", extName)
|
||||
}
|
||||
|
||||
props := &Properties{}
|
||||
props.Parse(desc.Tag)
|
||||
|
||||
typ := reflect.TypeOf(desc.ExtensionType)
|
||||
if err := p.checkForColon(props, typ); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
rep := desc.repeated()
|
||||
|
||||
// Read the extension structure, and set it in
|
||||
// the value we're constructing.
|
||||
var ext reflect.Value
|
||||
if !rep {
|
||||
ext = reflect.New(typ).Elem()
|
||||
} else {
|
||||
ext = reflect.New(typ.Elem()).Elem()
|
||||
}
|
||||
if err := p.readAny(ext, props); err != nil {
|
||||
if _, ok := err.(*RequiredNotSetError); !ok {
|
||||
return err
|
||||
}
|
||||
reqFieldErr = err
|
||||
}
|
||||
ep := sv.Addr().Interface().(Message)
|
||||
if !rep {
|
||||
SetExtension(ep, desc, ext.Interface())
|
||||
} else {
|
||||
old, err := GetExtension(ep, desc)
|
||||
var sl reflect.Value
|
||||
if err == nil {
|
||||
sl = reflect.ValueOf(old) // existing slice
|
||||
} else {
|
||||
sl = reflect.MakeSlice(typ, 0, 1)
|
||||
}
|
||||
sl = reflect.Append(sl, ext)
|
||||
SetExtension(ep, desc, sl.Interface())
|
||||
}
|
||||
if err := p.consumeOptionalSeparator(); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// This is a normal, non-extension field.
|
||||
name := tok.value
|
||||
var dst reflect.Value
|
||||
fi, props, ok := structFieldByName(sprops, name)
|
||||
if ok {
|
||||
dst = sv.Field(fi)
|
||||
} else if oop, ok := sprops.OneofTypes[name]; ok {
|
||||
// It is a oneof.
|
||||
props = oop.Prop
|
||||
nv := reflect.New(oop.Type.Elem())
|
||||
dst = nv.Elem().Field(0)
|
||||
field := sv.Field(oop.Field)
|
||||
if !field.IsNil() {
|
||||
return p.errorf("field '%s' would overwrite already parsed oneof '%s'", name, sv.Type().Field(oop.Field).Name)
|
||||
}
|
||||
field.Set(nv)
|
||||
}
|
||||
if !dst.IsValid() {
|
||||
return p.errorf("unknown field name %q in %v", name, st)
|
||||
}
|
||||
|
||||
if dst.Kind() == reflect.Map {
|
||||
// Consume any colon.
|
||||
if err := p.checkForColon(props, dst.Type()); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Construct the map if it doesn't already exist.
|
||||
if dst.IsNil() {
|
||||
dst.Set(reflect.MakeMap(dst.Type()))
|
||||
}
|
||||
key := reflect.New(dst.Type().Key()).Elem()
|
||||
val := reflect.New(dst.Type().Elem()).Elem()
|
||||
|
||||
// The map entry should be this sequence of tokens:
|
||||
// < key : KEY value : VALUE >
|
||||
// However, implementations may omit key or value, and technically
|
||||
// we should support them in any order. See b/28924776 for a time
|
||||
// this went wrong.
|
||||
|
||||
tok := p.next()
|
||||
var terminator string
|
||||
switch tok.value {
|
||||
case "<":
|
||||
terminator = ">"
|
||||
case "{":
|
||||
terminator = "}"
|
||||
default:
|
||||
return p.errorf("expected '{' or '<', found %q", tok.value)
|
||||
}
|
||||
for {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value == terminator {
|
||||
break
|
||||
}
|
||||
switch tok.value {
|
||||
case "key":
|
||||
if err := p.consumeToken(":"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.readAny(key, props.mkeyprop); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.consumeOptionalSeparator(); err != nil {
|
||||
return err
|
||||
}
|
||||
case "value":
|
||||
if err := p.checkForColon(props.mvalprop, dst.Type().Elem()); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.readAny(val, props.mvalprop); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.consumeOptionalSeparator(); err != nil {
|
||||
return err
|
||||
}
|
||||
default:
|
||||
p.back()
|
||||
return p.errorf(`expected "key", "value", or %q, found %q`, terminator, tok.value)
|
||||
}
|
||||
}
|
||||
|
||||
dst.SetMapIndex(key, val)
|
||||
continue
|
||||
}
|
||||
|
||||
// Check that it's not already set if it's not a repeated field.
|
||||
if !props.Repeated && fieldSet[name] {
|
||||
return p.errorf("non-repeated field %q was repeated", name)
|
||||
}
|
||||
|
||||
if err := p.checkForColon(props, dst.Type()); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Parse into the field.
|
||||
fieldSet[name] = true
|
||||
if err := p.readAny(dst, props); err != nil {
|
||||
if _, ok := err.(*RequiredNotSetError); !ok {
|
||||
return err
|
||||
}
|
||||
reqFieldErr = err
|
||||
}
|
||||
if props.Required {
|
||||
reqCount--
|
||||
}
|
||||
|
||||
if err := p.consumeOptionalSeparator(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if reqCount > 0 {
|
||||
return p.missingRequiredFieldError(sv)
|
||||
}
|
||||
return reqFieldErr
|
||||
}
|
||||
|
||||
// consumeExtName consumes extension name or expanded Any type URL and the
|
||||
// following ']'. It returns the name or URL consumed.
|
||||
func (p *textParser) consumeExtName() (string, error) {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return "", tok.err
|
||||
}
|
||||
|
||||
// If extension name or type url is quoted, it's a single token.
|
||||
if len(tok.value) > 2 && isQuote(tok.value[0]) && tok.value[len(tok.value)-1] == tok.value[0] {
|
||||
name, err := unquoteC(tok.value[1:len(tok.value)-1], rune(tok.value[0]))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, p.consumeToken("]")
|
||||
}
|
||||
|
||||
// Consume everything up to "]"
|
||||
var parts []string
|
||||
for tok.value != "]" {
|
||||
parts = append(parts, tok.value)
|
||||
tok = p.next()
|
||||
if tok.err != nil {
|
||||
return "", p.errorf("unrecognized type_url or extension name: %s", tok.err)
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, ""), nil
|
||||
}
|
||||
|
||||
// consumeOptionalSeparator consumes an optional semicolon or comma.
|
||||
// It is used in readStruct to provide backward compatibility.
|
||||
func (p *textParser) consumeOptionalSeparator() error {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value != ";" && tok.value != "," {
|
||||
p.back()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *textParser) readAny(v reflect.Value, props *Properties) error {
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value == "" {
|
||||
return p.errorf("unexpected EOF")
|
||||
}
|
||||
|
||||
switch fv := v; fv.Kind() {
|
||||
case reflect.Slice:
|
||||
at := v.Type()
|
||||
if at.Elem().Kind() == reflect.Uint8 {
|
||||
// Special case for []byte
|
||||
if tok.value[0] != '"' && tok.value[0] != '\'' {
|
||||
// Deliberately written out here, as the error after
|
||||
// this switch statement would write "invalid []byte: ...",
|
||||
// which is not as user-friendly.
|
||||
return p.errorf("invalid string: %v", tok.value)
|
||||
}
|
||||
bytes := []byte(tok.unquoted)
|
||||
fv.Set(reflect.ValueOf(bytes))
|
||||
return nil
|
||||
}
|
||||
// Repeated field.
|
||||
if tok.value == "[" {
|
||||
// Repeated field with list notation, like [1,2,3].
|
||||
for {
|
||||
fv.Set(reflect.Append(fv, reflect.New(at.Elem()).Elem()))
|
||||
err := p.readAny(fv.Index(fv.Len()-1), props)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tok := p.next()
|
||||
if tok.err != nil {
|
||||
return tok.err
|
||||
}
|
||||
if tok.value == "]" {
|
||||
break
|
||||
}
|
||||
if tok.value != "," {
|
||||
return p.errorf("Expected ']' or ',' found %q", tok.value)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
// One value of the repeated field.
|
||||
p.back()
|
||||
fv.Set(reflect.Append(fv, reflect.New(at.Elem()).Elem()))
|
||||
return p.readAny(fv.Index(fv.Len()-1), props)
|
||||
case reflect.Bool:
|
||||
// true/1/t/True or false/f/0/False.
|
||||
switch tok.value {
|
||||
case "true", "1", "t", "True":
|
||||
fv.SetBool(true)
|
||||
return nil
|
||||
case "false", "0", "f", "False":
|
||||
fv.SetBool(false)
|
||||
return nil
|
||||
}
|
||||
case reflect.Float32, reflect.Float64:
|
||||
v := tok.value
|
||||
// Ignore 'f' for compatibility with output generated by C++, but don't
|
||||
// remove 'f' when the value is "-inf" or "inf".
|
||||
if strings.HasSuffix(v, "f") && tok.value != "-inf" && tok.value != "inf" {
|
||||
v = v[:len(v)-1]
|
||||
}
|
||||
if f, err := strconv.ParseFloat(v, fv.Type().Bits()); err == nil {
|
||||
fv.SetFloat(f)
|
||||
return nil
|
||||
}
|
||||
case reflect.Int32:
|
||||
if x, err := strconv.ParseInt(tok.value, 0, 32); err == nil {
|
||||
fv.SetInt(x)
|
||||
return nil
|
||||
}
|
||||
|
||||
if len(props.Enum) == 0 {
|
||||
break
|
||||
}
|
||||
m, ok := enumValueMaps[props.Enum]
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
x, ok := m[tok.value]
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
fv.SetInt(int64(x))
|
||||
return nil
|
||||
case reflect.Int64:
|
||||
if x, err := strconv.ParseInt(tok.value, 0, 64); err == nil {
|
||||
fv.SetInt(x)
|
||||
return nil
|
||||
}
|
||||
|
||||
case reflect.Ptr:
|
||||
// A basic field (indirected through pointer), or a repeated message/group
|
||||
p.back()
|
||||
fv.Set(reflect.New(fv.Type().Elem()))
|
||||
return p.readAny(fv.Elem(), props)
|
||||
case reflect.String:
|
||||
if tok.value[0] == '"' || tok.value[0] == '\'' {
|
||||
fv.SetString(tok.unquoted)
|
||||
return nil
|
||||
}
|
||||
case reflect.Struct:
|
||||
var terminator string
|
||||
switch tok.value {
|
||||
case "{":
|
||||
terminator = "}"
|
||||
case "<":
|
||||
terminator = ">"
|
||||
default:
|
||||
return p.errorf("expected '{' or '<', found %q", tok.value)
|
||||
}
|
||||
// TODO: Handle nested messages which implement encoding.TextUnmarshaler.
|
||||
return p.readStruct(fv, terminator)
|
||||
case reflect.Uint32:
|
||||
if x, err := strconv.ParseUint(tok.value, 0, 32); err == nil {
|
||||
fv.SetUint(x)
|
||||
return nil
|
||||
}
|
||||
case reflect.Uint64:
|
||||
if x, err := strconv.ParseUint(tok.value, 0, 64); err == nil {
|
||||
fv.SetUint(x)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return p.errorf("invalid %v: %v", v.Type(), tok.value)
|
||||
}
|
||||
|
||||
// UnmarshalText reads a protocol buffer in Text format. UnmarshalText resets pb
|
||||
// before starting to unmarshal, so any existing data in pb is always removed.
|
||||
// If a required field is not set and no other error occurs,
|
||||
// UnmarshalText returns *RequiredNotSetError.
|
||||
func UnmarshalText(s string, pb Message) error {
|
||||
if um, ok := pb.(encoding.TextUnmarshaler); ok {
|
||||
err := um.UnmarshalText([]byte(s))
|
||||
return err
|
||||
}
|
||||
pb.Reset()
|
||||
v := reflect.ValueOf(pb)
|
||||
if pe := newTextParser(s).readStruct(v.Elem(), ""); pe != nil {
|
||||
return pe
|
||||
}
|
||||
return nil
|
||||
}
|
||||
+201
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "{}"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright {yyyy} {name of copyright owner}
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user