-
Notifications
You must be signed in to change notification settings - Fork 50
Feat/native meta contract #3742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Pavel Karpy <[email protected]>
|
|
||
| p2pCfg := c.appCfg.Meta.P2P | ||
|
|
||
| var chainCfg = config.Config{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i would like to be validated on what exact minimum config is enough for an RPC node to start. see applySidechainDefaults also below
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See the reviewcomments below.
| type Experimental struct { | ||
| ChainMetaData bool `mapstructure:"chain_meta_data"` | ||
| AllowEC bool `mapstructure:"allow_ec"` | ||
| ChainMetaData MetaChain `mapstructure:"chain_meta_data"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since it is experimental, i dont care if it is changed, but i may be wrong
| return nil | ||
| } | ||
|
|
||
| func (m MetaData) InitializeCache(_ interop.IsHardforkEnabled, _ uint32, _ *dao.Simple) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i saw cache usages, but i do not fully understand when and how it should be used. is it worth it? also, i was confused that some caches do not do slice.Clone on Copy. need comments about how and when to use it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is it worth it?
You may leave it empty for now, native contract may work with storage directly without cache. Cache is introduce to avoid storage access.
also, i was confused that some caches do not do slice.Clone on Copy
It may be done if the cache design allows it. For example, native Neo does never modify the slice itself, instead it replaces the cached slise by the new slice on every change, so there's no pointing in copying this slice. Ref. https://github.com/nspcc-dev/neo-go/blob/993eaaeaf6faef5d66647212bfa4ba9618ee7b2f/pkg/core/native/native_neo.go#L140-L141.
how and when to use it
The main usage scenario is: initialize cache on node start based on the contract storage state of the latest stored block. If some storage entry is changed (e.g. via setter contract method), then update both storage entry and cached entry. Use cache in getters. Changes applied to the native cache are being persisted or discarded along with the rest of contract storage changes made by transaction.
| if ic.Tx != nil { | ||
| txHash = ic.Tx.Hash().StringLE() | ||
| } | ||
| ic.Log.Info(runtime.SystemRuntimeLogMessage, append(fields, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do not know if there are any agreements on logs. there is a system call about them, not sure if it is ok to log smth in a free manner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may, but you likely don't need this log, it's mostly useful for debugging and I'd say it shouldn't be included into the final contract version. It should either be a proper notification for actions that are important for another parts of the system or shouldn't be logged at all.
Also, just for the record, you need AllowNotify for the method that uses this syscall (which is not presented in the current implementation).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's mostly useful for debugging
yeah, currently use it for debugging, will drop it later
| if !ok { | ||
| panic(fmt.Errorf("unexpected second argument value: %T expected, %T given", sigsVectors, args[1].Value())) | ||
| } | ||
| var sigVectors = make([][][]byte, 0, len(sigsVectors)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@roman-khimov, i believe that it is possible to make it a little bit easier and be an array of multisignatures REP out of len(placementVectors[i]), but not fully understand if it is really an easier way to do it (or possible at all)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kinda optimization, can be done a bit later.
| } | ||
|
|
||
| cnrListRaw := ic.DAO.GetStorageItem(m.ID, append([]byte{containerPlacementPrefix}, cID...)) | ||
| placementI, err := stackitem.Deserialize(cnrListRaw) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i saw two versions of serialization: this one and ic.DAO.GetItemCtx().Serialize. i believe they do the same, but i would like to know exactly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ic.DAO.GetItemCtx().Serialize uses DAO's serialization context with custom/default serialization limits. You need to use native.PutConvertibleToDAO, I'll create a PR for that in NeoGo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ref. nspcc-dev/neo-go#4118.
| } | ||
| }() | ||
|
|
||
| rpcServer := rpcsrv.New(chain, cfg.ApplicationConfiguration.RPC, netServer, nil, log.With(zap.String("subcomponent", "rpc server")), chErr) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is the most efficient (and correct at the same time) way to add a transaction to an RPC/Notary service's pool? now we have SNs that have local RPC and IRs that have local consensus. do i still need to use smth like rpcclient.NewInternal to operate with pools?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may directly use this API: https://github.com/nspcc-dev/neo-go/blob/993eaaeaf6faef5d66647212bfa4ba9618ee7b2f/pkg/network/server.go#L1372
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for Notary -- yes, but what about RPC? or maybe i need to use (bc *Blockchain) PoolTx?
| md = native.NewMethodAndPrice(m.removeMetaContainer, 0, callflag.WriteStates) | ||
| m.AddMethod(md, desc) | ||
|
|
||
| desc = native.NewDescriptor("updateContainerList", smartcontract.VoidType, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@roman-khimov, i do believe that having a chain locally and considering it as a special case for metadata makes it possible to have bigger limits without taking into account RPC limits and other public worries. Previously, we updated container lists in two stages: AddNextEpochNodes and CommitContainerListUpdate, now it is an independent sidechain, it is much easier to maintain a single method and container list. i am not sure what exact limit will be the first in this case, and if it is possible at all
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/nspcc-dev/neofs-api/blob/9da2bbb9b7048af96be3ca1ad9fb3e86fb1f1b28/netmap/types.proto#L118
We're ok with a single call.
| c.internalErr <- fmt.Errorf("meta data service error: %w", err) | ||
| } | ||
| })) | ||
| //if c.cfgMorph.client == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it supposed to be commented out?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not, it is a temporary migration/merging of old and new meta services for SNs
| }, | ||
| Magic: fsChainProtocol.Network + 1, | ||
| InitialGASSupply: fsChainProtocol.InitialGasDistribution, | ||
| P2PNotaryRequestPayloadPoolSize: 1000, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be removed, will be set to default by https://github.com/nspcc-dev/neo-go/blob/993eaaeaf6faef5d66647212bfa4ba9618ee7b2f/pkg/core/blockchain.go#L291.
| Genesis: config.Genesis{ | ||
| MaxTraceableBlocks: fsChainProtocol.MaxTraceableBlocks, | ||
| MaxValidUntilBlockIncrement: fsChainProtocol.MaxValidUntilBlockIncrement, | ||
| Roles: nil, // not needed? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure, but we may want to preset state validators to be able to use NeoFS-based state sync right from the chain start.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, don't you use Notaries for this chain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, don't you use Notaries for this chain?
i do, but the notary service starts only on the IR side, and this is SN's RPC node that does not receive notary requests. looks like this is a mistake, but for some reason it starts and fetches blocks ok
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Genesis configuration should match exactly on all nodes of the network, including Genesis.Roles.
i do
So you probably have to setup Notary nodes via Genesis.Roles.
| Addresses: c.appCfg.Meta.RPC.Listen, | ||
| } | ||
| rpcConfig.MaxGasInvoke = fixedn.Fixed8FromInt64(int64(rpcCfgRead.MaxGasInvoke)) | ||
| rpcConfig.MaxIteratorResultItems = 100 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use https://github.com/nspcc-dev/neo-go/blob/2aef3337eed2994ec756bcd145502ce82eb7e7c1/pkg/config/config.go#L28 if you need to set the default. Keep it empty in case if you're OK with server defaults, the config will be sanitized on RPC server construction, ref. https://github.com/nspcc-dev/neo-go/blob/2aef3337eed2994ec756bcd145502ce82eb7e7c1/pkg/services/rpcsrv/server.go#L286.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose it's the answer to https://github.com/nspcc-dev/neofs-node/pull/3742/changes#r2628944024.
| desc := native.NewDescriptor("submitObjectPut", smartcontract.VoidType, | ||
| manifest.NewParameter("metaInformation", smartcontract.ByteArrayType), | ||
| manifest.NewParameter("signatures", smartcontract.ArrayType)) | ||
| md := native.NewMethodAndPrice(m.submitObjectPut, 0, callflag.States|callflag.AllowNotify) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it free?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tbh, i am not sure what economics we are supposed to use there. as always, i tend to think that fewer balance problems are better for storage maintenance
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on our Element conversation, I'd suggest to charge at least some GAS for the execution. The price may be relatively aligned with some similar native method of other contract. The problem of balance will be solved with custom Gas implementation.
| md = native.NewMethodAndPrice(m.registerMetaContainer, 0, callflag.WriteStates) | ||
| m.AddMethod(md, desc) | ||
|
|
||
| desc = native.NewDescriptor("removeMetaContainer", smartcontract.VoidType, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/removeMetaContainer/unregisterMetaContainer
Because you have registerMetaContainer, names should be aligned.
| var txHash string | ||
| if ic.Tx != nil { | ||
| txHash = ic.Tx.Hash().StringLE() | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use container hash instead, it's always non-emty, but it's either block or tx.
| if ic.Tx != nil { | ||
| txHash = ic.Tx.Hash().StringLE() | ||
| } | ||
| ic.Log.Info(runtime.SystemRuntimeLogMessage, append(fields, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may, but you likely don't need this log, it's mostly useful for debugging and I'd say it shouldn't be included into the final contract version. It should either be a proper notification for actions that are important for another parts of the system or shouldn't be logged at all.
Also, just for the record, you need AllowNotify for the method that uses this syscall (which is not presented in the current implementation).
| "go.uber.org/zap" | ||
| ) | ||
|
|
||
| func (m MetaData) removeMetaContainer(ic *interop.Context, args []stackitem.Item) stackitem.Item { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define these methods on pointer.
| newContracts = append(newContracts, contract) | ||
| case *native.Std, *native.Crypto, *native.Oracle: | ||
| default: | ||
| panic(fmt.Sprintf("unexpected native contract found: %T", contract)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will panic after nspcc-dev/neo-go#4113.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think it is a feature, if we need more contracts to support at some point in the future (who knows), we'd better know it from tests immediately
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Panic is OK to me. Just fetch the latest NeoGo master to properly handle the new native.
| func (m MetaData) checkCommitteeWitness(ic *interop.Context) error { | ||
| h := m.neo.GetCommitteeAddress(ic.DAO) | ||
| if ok, err := runtime.CheckHashedWitness(ic, h); err != nil || !ok { | ||
| return native.ErrInvalidWitness | ||
| } | ||
|
|
||
| return nil | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use (INEO).CheckCommittee instead, that's it.
| } | ||
|
|
||
| ic.DAO.DeleteStorageItem(m.ID, append([]byte{metaContainersPrefix}, cID...)) | ||
| logMessage(ic, "removed meta container", zap.String("cID", base58.Encode(cID))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, either send a proper notification or don't log at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto below.
| metaContainersPrefix = '1' | ||
| containerPlacementPrefix = '2' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| metaContainersPrefix = '1' | |
| containerPlacementPrefix = '2' | |
| metaContainersPrefix = 0x01 | |
| containerPlacementPrefix = 0x02 |
That's it, it's more native than 49/50 and will be easier to debug with some findstorage.
| case *native.NEO: | ||
| neoContract = contract.(native.INEO) | ||
| newContracts = append(newContracts, neoContract) | ||
| case *native.Management, *native.Ledger, *native.GAS, *native.Policy, *native.Designate, *native.Notary: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're supposed to have a custom (and simplified) Neo/Gas implementations. Also, do you need Notary at all for this chain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, do you need Notary at all for this chain?
alphabet nodes are gonna collect notary signatures for sure, yes
We're supposed to have a custom (and simplified) Neo/Gas implementations
has not found what exact simplifications are required for us, that is why defaults for now. will think about it later
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose we need a separate issue to simplify the set of native contracts for meta chain.
| switch typ.Int64() { | ||
| case 0, 1, 2, 3, 4: // regular, tombstone, storage group, lock, link | ||
| default: | ||
| panic("incorrect object type") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Include typ into panic.
| newContracts = append(newContracts, neoContract) | ||
| case *native.Management, *native.Ledger, *native.GAS, *native.Policy, *native.Designate, *native.Notary: | ||
| newContracts = append(newContracts, contract) | ||
| case *native.Std, *native.Crypto, *native.Oracle: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the current set of contracts it's OK, but if you're going to use custom implementations of existing contracts, then you need to properly initialize dependencies of default natives.
| } | ||
| }() | ||
|
|
||
| rpcServer := rpcsrv.New(chain, cfg.ApplicationConfiguration.RPC, netServer, nil, log.With(zap.String("subcomponent", "rpc server")), chErr) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may directly use this API: https://github.com/nspcc-dev/neo-go/blob/993eaaeaf6faef5d66647212bfa4ba9618ee7b2f/pkg/network/server.go#L1372
PutConvertibleToDAO and GetConvertibleFromDAO can be used by custom native contracts, ref. nspcc-dev/neofs-node#3742 (comment). Signed-off-by: Anna Shaleva <[email protected]>
| p2pCfg := c.appCfg.Meta.P2P | ||
|
|
||
| var chainCfg = config.Config{ | ||
| ProtocolConfiguration: config.ProtocolConfiguration{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for the record: I don't see Hardforks being set. It means that all stable hardforks are active starting from genesis. It's expected and correct behaviour for meta chain, I'm just letting you know that.
However, NeoGo is continuously adding more hardforks, so if some new hardfork is added, then it may affect states after subsequent NeoGo dependency update in meta chain. So to me, it's better to explicitly set all desired hardfork heights to 0 here.
putConvertibleToDAO and getConvertibleFromDAO can be used by custom native contracts, ref. nspcc-dev/neofs-node#3742 (comment). Move these helpers to (*DAO) so that everyone can use them. Signed-off-by: Anna Shaleva <[email protected]>
New application configs example: nspcc-dev/neofs-dev-env#334