Don't use mnl attr helpers, we're trying to remove the libmnl
dependency. Create both signed and unsigned helpers, libmnl
had unsigned helpers, so code generator no longer needs
the mnl_type() hack.
The new helpers are written from first principles, but are
hopefully not too buggy.
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Link: https://lore.kernel.org/r/20240227223032.1835527-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The temporary auto-int helpers are not really correct.
We can't treat signed and unsigned ints the same when
determining whether we need full 8B. I realized this
before sending the patch to add support in libmnl.
Unfortunately, that patch has not been merged,
so time to fix our local helpers. Use the mnl* name
for now, subsequent patches will address that.
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240227223032.1835527-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We never increment the group number iterator, so all groups
get recorded into index 0 of the mcast_groups[] array.
As a result YNL can only handle using the last group.
For example using the "netdev" sample on kernel with
page pool commands results in:
$ ./samples/netdev
YNL: Multicast group 'mgmt' not found
Most families have only one multicast group, so this hasn't
been noticed. Plus perhaps developers usually test the last
group which would have worked.
Fixes: 86878f14d71a ("tools: ynl: user space helpers")
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Link: https://lore.kernel.org/r/20240226214019.1255242-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Since commit 7c59c9c8f202 ("tools: ynl: generate code for ovs families")
we need relatively recent OvS headers to get YNL to compile.
Add the direct include workaround to fix compilation on less
up-to-date OSes like CentOS 9.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240226225806.1301152-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
devlink and ethtool have a trailing _ in the header guard. I must have
copy/pasted it into new guards, assuming it's a headers_install artifact.
This fixes build if system headers are old.
Fixes: 8f109e91b852 ("tools: ynl: include dpll and mptcp_pm in C codegen")
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240222234831.179181-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
During decoding of messages coming from kernel, attribute values are
converted to enum names in case the attribute type is enum of bitfield32.
However, when user constructs json message, he has to pass plain scalar
values. See "state" "selector" and "value" attributes in following
examples:
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/dpll.yaml --do pin-set --json '{"id": 0, "parent-device": {"parent-id": 0, "state": 1}}'
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/devlink.yaml --do port-set --json '{"bus-name": "pci", "dev-name": "0000:08:00.1", "port-index": 98304, "port-function": {"caps": {"selector": 1, "value": 1 }}}'
Allow user to pass strings containing enum names, convert them to scalar
values to be encoded into Netlink message:
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/dpll.yaml --do pin-set --json '{"id": 0, "parent-device": {"parent-id": 0, "state": "connected"}}'
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/devlink.yaml --do port-set --json '{"bus-name": "pci", "dev-name": "0000:08:00.1", "port-index": 98304, "port-function": {"caps": {"selector": ["roce-bit"], "value": ["roce-bit"] }}}'
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240222134351.224704-4-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
As a preparation to handle enums for scalar values, unify the processing
of all scalar types in a single elif statement.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://lore.kernel.org/r/20240222134351.224704-3-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The flag attr presence in Netlink message indicates value "true",
if it is missing in the message it means "false".
Allow user to specify attrname with value "true"/"false"
in json for flag attrs, treat "false" value properly.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240222134351.224704-2-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Make sure to free the already-parsed mcast_groups if
we don't get an ack from the kernel when reading family info.
This is part of the ynl_sock_create() error path, so we won't
get a call to ynl_sock_destroy() to free them later.
Fixes: 86878f14d71a ("tools: ynl: user space helpers")
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Link: https://lore.kernel.org/r/20240220161112.2735195-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
There is one common error handler in ynl - ynl_cb_error().
It expects priv to be a pointer to struct ynl_parse_arg AKA yarg.
To avoid potential crashes if we encounter a stray NLMSG_ERROR
always pass yarg as priv (or a struct which has it as the first
member).
ynl_cb_null() has a similar problem directly - it expects yarg
but priv passed by the caller is ys.
Found by code inspection.
Fixes: 86878f14d71a ("tools: ynl: user space helpers")
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Link: https://lore.kernel.org/r/20240220161112.2735195-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
If message contains unknown attribute and user passes
"--process-unknown" command line option, _decode() gets called with space
arg set to None. In that case, attr_space variable is not initialized
used which leads to following trace:
Traceback (most recent call last):
File "./tools/net/ynl/cli.py", line 77, in <module>
main()
File "./tools/net/ynl/cli.py", line 68, in main
reply = ynl.dump(args.dump, attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tools/net/ynl/lib/ynl.py", line 909, in dump
return self._op(method, vals, [], dump=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tools/net/ynl/lib/ynl.py", line 894, in _op
rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tools/net/ynl/lib/ynl.py", line 639, in _decode
self._rsp_add(rsp, attr_name, None, self._decode_unknown(attr))
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tools/net/ynl/lib/ynl.py", line 569, in _decode_unknown
return self._decode(NlAttrs(attr.raw), None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tools/net/ynl/lib/ynl.py", line 630, in _decode
search_attrs = SpaceAttrs(attr_space, rsp, outer_attrs)
^^^^^^^^^^
UnboundLocalError: cannot access local variable 'attr_space' where it is not associated with a value
Fix this by moving search_attrs assignment under the if statement
above it to make sure attr_space is initialized.
Fixes: bf8b832374fb ("tools/net/ynl: Support sub-messages in nested attribute spaces")
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Multi-attr elements could not be encoded because of missing logic in the
ynl code. Enable encoding of these attributes by checking if the
attribute is a multi-attr and if the value to be processed is a list.
This has been tested both with the taprio and ets qdisc which contain
this kind of attributes.
Signed-off-by: Alessandro Marcolini <alessandromarcolini99@gmail.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/c5bc9f5797168dbf7a4379c42f38d5de8ac7f38a.1706962013.git.alessandromarcolini99@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Instead of listing the genetlink families that we want to codegen
for, always codegen for everyone. We can add an opt-out later but
it seems like most families are not causing any issues, and yet
folks forget to add them to the Makefile.
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240202004926.447803-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add ovs_flow, ovs_vport and ovs_datapath to the families supported
in C. ovs-flow has some circular nesting which is fun to deal with,
but the necessary support has been added already in the previous
release cycle.
Add a sample that proves that dealing with fixed headers does
actually work correctly.
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240202004926.447803-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The DPLL and mptcp_pm families are pretty clean, and YNL C codegen
supports them fully with no changes. Add them to user space codegen
so that C samples can be written, and we know immediately if changes
to these families require YNL codegen work.
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240202004926.447803-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Extend the ynl doc generator to include type information for struct
members, ignoring the pad type.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-13-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Make it possible for struct definitions to reference other struct
definitions ofr binary members. For example, the tbf qdisc uses this
struct definition for its parms attribute:
-
name: tc-tbf-qopt
type: struct
members:
-
name: rate
type: binary
struct: tc-ratespec
-
name: peakrate
type: binary
struct: tc-ratespec
-
name: limit
type: u32
-
name: buffer
type: u32
-
name: mtu
type: u32
This adds the necessary schema changes and adds nested struct encoding
and decoding to ynl.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-11-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The formatted_string() class method was in NlAttr so that it could be
accessed by NlAttr.as_struct(). Now that as_struct() has been removed,
move formatted_string() to YnlFamily as an internal helper method.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-10-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Refactor the _fixed_header_size() method to be _struct_size() so that
naming is consistent with _encode_struct() and _decode_struct().
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-9-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
_decode_fixed_header() and NlAttr.as_struct() both implemented struct
decoding logic. Deduplicate the code into newly named _decode_struct()
method.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-8-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support for defaulting binary byte arrays to all zeros as well as
defaulting scalar values to 0 when encoding input parameters.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-7-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add sub-message encoding to ynl. This makes it possible to create
tc qdiscs and other polymorphic netlink objects.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-6-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Refactor the fixed header encoding into a separate _encode_struct method
so that it can be reused for fixed headers in sub-messages and for
encoding structs.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-5-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Sub-message selectors could only be resolved using values from the
current nest level. Enable value lookup in outer scopes by using
collections.ChainMap to implement an ordered lookup from nested to
outer scopes.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-3-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The ynl cli currently emits python pretty printed structures which is
hard to consume. Add a new --output-json argument to emit JSON.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-2-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The output from ynl-gen-rst.py has extra indentation that causes extra
<blockquote> elements to be generated in the HTML output.
Reduce the indentation so that sphinx doesn't generate unnecessary
<blockquote> elements.
Reviewed-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231215093720.18774-14-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The generated .rst for attribute-sets currently uses a sub-sub-heading
for each attribute, with the attribute name in bold. This makes
attributes stand out more than the attribute-set sub-headings they are
part of.
Remove the bold markup from attribute sub-sub-headings.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231215093720.18774-13-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The index of netlink specs was being generated unsorted. Sort the output
before generating the index entries.
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231215093720.18774-12-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The tc netlink-raw family needs binary and pad types for several
qopt C structs. Add support for them to ynl.
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231215093720.18774-6-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Implement the 'sub-message' attribute type in ynl.
Encode support is not yet implemented. Support for sub-message selectors
at a different nest level from the key attribute is not yet supported.
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231215093720.18774-5-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Use expression formatting that conforms to the python style guide.
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231215093720.18774-2-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We avoid printing forward declarations and prototypes for most
types by sorting things topologically. But if structs nest we
do need the forward declarations, there's no other way.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-9-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
To avoid infinite nesting store recursive structs by pointer.
If recursive struct is placed in the op directly - the first
instance can be stored by value. That makes the code much
less of a pain for majority of practical uses.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-8-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We try to keep the structures and helpers "topologically sorted",
to avoid forward declarations. When recursive nests are at play
we need to sort twice, because structs which end up being marked
as recursive will get a full set of forward declarations, so we
should ignore them for the purpose of sorting.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Track which nests are recursive. Non-recursive nesting gets
rendered in C as directly nested structs. For recursive
ones we need to put a pointer in, rather than full struct.
Track this information, no change to generated code, yet.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Fill in more empty handlers for TypeUnused. When 'unused'
attr gets specified in a nested set we have to cleanly
skip it during code generation.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Support genetlink families using simple fixed headers.
Assume fixed header is identical for all ops of the family for now.
Fixed headers are added to the request and reply structs as a _hdr
member, and copied to/from netlink messages appropriately.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit 30c902001534 ("tools: ynl-gen: use enum name from the spec")
added pre-cooked user type for enums. Use it to fix ignoring
enum-name provided in the spec.
This changes a type in struct ethtool_tunnel_udp_entry but is
generally inconsequential for current families.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The code gen generates a prototype for dump request free
in the header, but no implementation in the source.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ynl.h has a growing amount of "internal" stuff, which may confuse
users who try to take a look at the external API. Currently the
internals are at the bottom of the file with a banner in between,
but this arrangement makes it hard to add external APIs / inline
helpers which need internal definitions.
Move internals to a separate header.
Link: https://lore.kernel.org/r/20231202211225.342466-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
If kernel didn't give use any meaningful error - print
a strerror() to the ynl error message.
Reviewed-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Link: https://lore.kernel.org/r/20231202211310.342716-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit 1768d8a767f8 ("tools/net/ynl: Add support for create flags")
added support for setting legacy netlink CRUD flags on netlink
messages (NLM_F_REPLACE, _EXCL, _CREATE etc.).
Most of genetlink won't need these, don't force callers to pass
in an empty argument to each do() call.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231202211005.341613-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The ynl-generated user space C code is already above 25kLoC
and is growing.
The initial reason to commit these files was to make reviewing changes
to the generator easier. Unfortunately, it has the opposite effect on
reviewing changes to specs, and we get far more changes to specs
than to the generator.
Uncommit those fails, as they are generated on the fly as needed.
netdev patchwork now runs a script on each series to create a diff
of generated code on the fly, for the rare cases when looking at
it is helpful:
https://github.com/kuba-moo/nipa/blob/master/tests/series/ynl/ynl.sh
Suggested-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add support in netlink spec(netdev.yaml) for interrupt number
among the NAPI attributes. Add code generated from the spec.
Signed-off-by: Amritha Nambiar <amritha.nambiar@intel.com>
Reviewed-by: Sridhar Samudrala <sridhar.samudrala@intel.com>
Link: https://lore.kernel.org/r/170147334210.5260.18178387869057516983.stgit@anambiarhost.jf.intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
