speculoos.function-specs

This namespace provides facilities to apply specifications to and validate functions. Roughly speaking, trying to replicate the instrumentation and testing capabilities of spec.alpha .

Speculoos explores a few different styles of function specification:

  1. Explicitly providing the specification during explicit validation.
  2. Implicitly providing the specification via metadata, explicitly validating.
  3. Implicitly providing the specification via metadata, implicitly validating via instrumentation.

Warning: Several actions performed by functions in this namespace mutate state, and are brittle. Use this namespace with the understanding that it is very much a proof-of-concept.

assoc-metadata-f!

(assoc-metadata-f! f k v)

Associate metadata key k, value v to the var of function f. Returns the metadata map. See dissoc-metadata-f! for the inverse operation.

Example:

(defn foo [] true)

(assoc-metadata-f! foo :assoced 'bar!)

(:assoced (meta #'foo)) ;; => 'bar!

dissoc-metadata-f!

(dissoc-metadata-f! f k)

Dissociate metadata key k from the var of function f. Returns the metadata map. See assoc-metadata-f! for the inverse operation.

Example:

(defn foo [] true)

(assoc-metadata-f! foo :assoced 'bar!)

(:assoced (meta #'foo)) ;; => 'bar!

(dissoc-metadata-f! foo :assoced)

(:assoced (meta #'foo)) ;; => nil

exercise-fn

(exercise-fn f)(exercise-fn f n)

Exercises the function f by applying it to n (default 10) generated samples of its scalar argument specification, residing at :speculoos/arg-scalar-spec in its metadata.

Returns a sequence of tuples of [args ret]. Does not currently handle higher-order-functions.

If n is :canonical, only one data set is produced, consisting of the predicates’ canonical values.

See data-from-spec for details on predicates.

Example:

(defn foo [x kw] (str (+ 100 x) "is not equivalent to " kw))

(foo 5 :bar) ;; "105 is not equivalent to :bar"

(inject-specs! foo {:speculoos/arg-scalar-spec [int? keyword?]})

(exercise-fn foo 3)
;; => ([[76 :i-:!7S] "176 is not equivalent to :i-:!7S"]
;;     [[-381 :W] "-281 is not equivalent to :W"]
;;     [[-940 :LS1-:i] "-840 is not equivalent to :LS1-:i"])

Examples with canonical values:

(defn bar
  {:speculoos/arg-scalar-spec [int? ratio? float?]}
  [x y z]
  (+ x y z))

(exercise-fn bar :canonical)
;; => ([[42 22/7 1.23] 46.372857142857136])

inject-specs!

(inject-specs! f specs)

Given function f, associates scalar and collection specifications for arguments and return values. specs is a map, whose only recognized pseudo-qualified keys are contained in recognized-spec-keys. No warnings are given for key-vals that are not recognized and thus not injected. Returns nil. See unject-specs! for the inverse operation.

Example:

(defn foo [] true)

(inject-specs! foo {:speculoos/ret-scalar-spec boolean?}) ;; => nil

(:speculoos/ret-scalar-spec (meta #'foo)) ;; => boolean?

instrument

(instrument f)

Wrap function f such that invoking f implicitly performs argument validation before and return value validation after execution. See wrapping-fn for details.

Implemented by:

  1. Adding an entry into the instrumentation registry.
  2. Altering f’s root var to wrapped version of itself.

The inverse operation is provided by unstrument.

Warning: This implementation is experimental and brittle. Since it relies on mutating vars, it is sensitive to order of invocation and multiple invocations.

Examples:

(defn add-n-subtract [x y] [(+ x y) (- x y)])

;; `instrument` requires specifications be added *before* instrumentation
(inject-specs! add-n-subtract {:speculoos/arg-scalar-spec [int? ratio?]})

(instrument add-n-subtract)

;; while instrumented, each invocation automatically validates any supplied specification
;; non-satisfied predicates are send to *out*
(with-out-str (add-n-subtract 5 2)) ;; printed to *out* ({:path [1], :datum 2, :predicate ratio?, :valid? false})

;; if all specifications are satisfied, the function returns a value
(add-n-subtract 5 3/2) ;; => [13/2 7/2]

;; remove instrumentation wrapper...
(unstrument add-n-subtract) ;; => {}

;; ...but metadata specifications remain
(:speculoos/arg-scalar-spec (meta #'add-n-subtract)) ;; => [int? ratio?]

;; function resumes normal behavior; non-satisfying arguments are consumed without note
(add-n-subtract 5 2) ;; => [7 3]
(add-n-subtract 5 3/2) ;; => [13/2 7/2]

recognized-spec-keys

A sequence that contains the only allowed pseudo-qualified keys to be added to, or referred from, a function’s metadata. Only governs behavior of utilities provided by this namespace, such as instrument, unstrument, validate-fn-with, and validate-fn. Does not affect anything outside this namespace.

unject-specs!

(unject-specs! f)

Given function f, dissociates any key-vals contained in recognized-spec-keys. Inverse operation provided by inject-specs!. Returns nil.

Example:

(defn foo [] true)

(inject-specs! foo {:speculoos/ret-scalar-spec boolean?}) ;; => nil

(:speculoos/ret-scalar-spec (meta #'foo)) ;; => boolean?

(unject-specs! foo) ;; => nil

(:speculoos/ret-scalar-spec (meta #'foo)) ;; => nil

unstrument

macro

(unstrument f)

Un-wrap function f to its original form that does not implicitly validate arguments nor return values.pre-fn nor post-fn. This is the inverse operation of instrument.

Compliments to whoever originally thought of ‘unstrument’.

validate-argument-return-relationship

(validate-argument-return-relationship arg ret rel)

Validates an argument/return relationship given argument sequence arg, function return value ret, and relationship rel, a map of {:path-argument … :path-return … :relationship-fn …}. :relationship-fn is a 2-arity function that presumably tests some relationship between a slice of arg, passed as the first argument to the relationship function, and a slice of ret, passed as the second argument to the relationship function. A nil path indicates a ‘bare’ scalar value (i.e., a non-collection). arg ought to always be a sequence.

Intended to validate the relationship between a function’s arguments and its return value.

Examples:

(defn doubled? [x y] (= y (* 2 x)))

;; 'bare' function return, uses path `nil`
(validate-argument-return-relationship [42] 84 {:path-argument [0]
                                                :path-return nil
                                                :relationship-fn doubled?})
;; => {:path-argument [0],
;;     :path-return nil,
;;     :relationship-fn doubled?,
;;     :datum-argument 42,
;;     :datum-return 84,
;;     :valid? true}


(defn reversed? [v1 v2] (= v2 (reverse v1)))

;; argument and return value fail to satisfy relationship
(validate-argument-return-relationship [1 2 3] [1 2 3] {:path-argument []
                                                        :path-return []
                                                        :relationship-fn reversed?})
;; => {:path-argument [],
;;     :path-return [],
;;     :relationship-fn reversed?,
;;     :datum-argument [1 2 3],
;;     :datum-return [1 2 3],
;;     :valid? false}

validate-fn

(validate-fn f & args)

Validates function f in the manner of validate-fn-with, except specifications are supplied by the function’s metadata, addressed by recognized-spec-keys.

Examples:

(defn foo [x s] (+ x (read-string s)))
(foo 7 "8") ;; 15

(def foo-spec {:speculoos/arg-scalar-spec [int? string?]
               :speculoos/ret-scalar-spec number?})

(inject-specs! foo foo-spec) ;; => nil

;; supplying valid arguments; function returns
(validate-fn foo 7 "8") ;; 15

;; supplying invalid argument (arg2 is not a string); yields a report
(validate-fn foo 7 8)
;; => ({:path [1], :datum 8, :predicate string?, :valid? false, :fn-spec-type :speculoos/argument}
;; foo was not able to yield a value, therefore the return was not validated

validate-fn-with

(validate-fn-with f specs & args)

Validate the scalar and collection aspects of arguments args, the return value, and the relationship between the arguments and return value, of function f. specs is a map with any permutation of recognized-spec-keys. Returns f’s results if specs are fully satisfied, otherwise, returns a report.

Note: Argument and return values satisfying the specifications does not guarantee that the function’s output is correct.

See validate-scalars and validate-collections for details about scalar and collection validation.

See validate-argument-return-relationship for details about validating relationships between the function’s argument and the function’s return value.

Example, validating scalars:

(defn foo [x y] (+ x y))

;; no specifications; return value passes through
(validate-fn-with foo {} 2 3)
;; => 5

;; all scalar specifications satisfied; return value passes through
(validate-fn-with foo
                  {:speculoos/arg-scalar-spec [int? int?]
                   :speculoos/ret-scalar-spec int?}
                  2 3)
;; => 5

;; one argument scalar specification and the return value scalar specification not satisfied
(validate-fn-with foo
                  {:speculoos/arg-scalar-spec [int? int?]
                   :speculoos/ret-scalar-spec int?}
                  2 3.3)
;; => ({:path [1], :datum 3.3, :predicate int?, :valid? false, :fn-spec-type :speculoos/argument}
;;     {:path nil, :datum 5.3, :predicate int?, :valid? false, :fn-spec-type :speculoos/return})

Example, validating argument/return value relationship:

;; function to validate
(defn broken-reverse [v] v)

;; function to test if the return collection is a correctly reversed version of the argument collection
(defn reversed? [v1 v2] (= v2 (reverse v1)))

;; yup, it's truly broken
(reversed? [11 22 33] (broken-reverse [11 22 33]))
;; => false

;; `broken-reverse` fails to satisfy the relationship function because it doesn't correctly reverse the argument collection
(validate-fn-with broken-reverse
                  {:speculoos/argument-return-relationships [{:path-argument [0]
                                                              :path-return []
                                                              :relationship-fn reversed?}]}
                  [11 22 33])
;; => ({:path-argument [0],
;;      :path-return [],
;;      :relationship-fn reversed?,
;;      :datum-argument [11 22 33],
;;      :datum-return [11 22 33],
;;      :valid? false,
;;      :fn-spec-type
;;      :speculoos/argument-return-relationship})

validate-higher-order-fn

(validate-higher-order-fn f & args)

Evaluates arbitrarily-deep, higher-order function f to exhaustion (i.e., until it yields a value that is not a function), validating arguments args and the return value. Halts on exceptions. Caller must supply sufficient &-arg vectors such that the final output could satisfy the return specification, if it exists.

Specifications are supplied in the top-level function’s metadata. All other lower-level function var metadata is ignored. Specifications must be a member of recognized-spec-keys. Specifications for the top-level function reside in the standard keys. Specifications for lower-level functions are nested into their respective tiers of the :speculoos/hof-specs key. If an argument collection specification is supplied for a lower-level function, a collection specification must be supplied for all functions ‘above’ that, even if those higher-level specifications are empty.

If every specification is satisfied, then the function’s evaluated value is returned, otherwise a sequence of invalidation reports.

Un-defined behavior if the final yield is nil or a function.

If you would call

(((f 1 2) :foo :bar) 'a 'b)

then the validation would be

(validate-higher-order-fn f [1 2] [:foo :bar] ['a 'b])

Use (with-meta f m) for ad hoc specification.

Example:

(defn foo [x y] (fn [w z] (* (+ x w) (- y z))))

(def bar (foo 8 11))
(bar 12 6) ;; 100

(def foo-spec {:speculoos/arg-scalar-spec [int? ratio?]
               :speculoos/hof-specs {:speculoos/arg-scalar-spec [int? ratio?]}})

(inject-specs! foo foo-spec) ;; nil

;; even though `foo` could consume these arguments and produce a value,
;; the arguments do not satisfy specification
(validate-higher-order-fn foo [8 11] [12 6])
;; => ({:path [0 1], :datum 11, :predicate ratio?, :valid? false, :fn-tier :speculoos/argument}
;;     {:path [1 1], :datum 6, :predicate ratio?, :valid? false, :fn-tier :speculoos/argument})

;; arguments satisfy specification, so `foo` returns the value
(validate-higher-order-fn foo [8 16/3] [12 1/3]) ;; 100N

wrapping-fn

(wrapping-fn f)

Low-level plumbing for instrument. See validate-fn-with for details.

Returns a function that will sequentially:

  1. Validate arguments with scalar specifications.
  2. Validate argument sequence with collection specification.
  3. Invoke function f.
  4. Validate return value with scalar specification.
  5. Validate return value with collection specification.
  6. Validate arguments versus return specifications.

Execution of the returned function will print non-satisfied specs to *out* and return the value produced by invoking f with the supplied arguments. At time of wrapping, function metadata must contain any desired specifications, possibly added by inject-specs!. Will not halt on non-valid specs.

Note: Arguments and return value satisfying their specification does not guarantee that the function’s output is correct.